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Introduction 


Clinical  practice  guidelines  and  protocols  are  being  applied  in  diverse  areas  including  policy 
development,  utilization  management,  education,  reference,  clinical  decision  support, 
conduct  of  clinical  trials,  and  workflow  facilitation.  Many  parties  are  engaged  in  developing 
guidelines,  an  arduous  task  with  much  redimdancy  and  overlap  among  the  resulting  products, 
but  there  is  little  standardization  to  facilitate  sharing  or  to  enable  adaptation  to  local  practice 
settings.  Yet  considerable  progress  has  been  made  on  developing  approaches  to  addressing 
these  issues,  and  standardized  approaches  for  guideline  representation  and  sharing  are  central 
to  these  efforts. 

To  address  these  issues,  a  workshop  titled  “Toward  a  Sharable  Guideline  Representation” 
was  organized  in  Boston,  MA  on  March  3-4, 2000.  This  workshop  focused  on: 

•  the  issues,  applications,  and  purposes  for  guideline  use  in  order  to  ensure  that  a 
standardized  representation  is  sufficiently  robust  to  address  these  purposes 

•  the  technical  requirements  for  such  a  representation 

•  the  establishment  of  a  multi-stakeholder  group  process  for  achieving  these  goals. 

This  workshop  brought  together  stakeholders  representing  academic  medical  informatics,  health 
care  payers,  professional  specialty  organizations,  health  care  providers,  the  Federal  government, 
and  the  health  care  information  industry,  from  both  the  US  and  abroad. 

Invited  speakers  and  discussion  leaders  facilitated  the  workshop  process.  Initial  plenary  talks 
provided  background  on  a  variety  of  work  and  accomplishments  to  date,  the  needs,  and  the 
charge  to  the  workshop  participants.  Breakout  groups  then  focused  on  issues  of  functionality, 
sharable  representation  approach,  and  organizational  issues  of  a  guideline-sharing  framework. 
The  output  of  the  meeting  was  a  set  of  position  statements,  a  proposed  organizational  structure, 
and  increased  collaboration  and  discussion  amongst  interested  parties  using  e-mail  discussion 
groups. 

Body 

The  workshop  was  organized  by 
Aziz  Boxwala,  MBBS,  PhD  (co-chair) 

Lucila  Ohno-Machado  MD,  PhD  (co-chair) 

Robert  A.  Greenes,  MD,  PhD 
Edward  H.  Shortliffe,  MD,  PhD 

The  agenda  consisted  of  two  plenary  sessions  followed  by  breakouts.  The  plenary  sessions  were 
intended  to  provide  an  overview  of  the  guideline  representation  efforts  to  date  and  to  have 
backgroimd  talks  on  related  topics. 

The  table  below  lists  the  presentations  in  the  first  plenary  session. 
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Robert  A.  Greenes 

Purpose  of  the  workshop 

Edward  H.  Shortliffe 

Motivation  for  structured  guidelines 

Aziz  A.  Boxwala 

Functional  requirements  of  guideline  representation  and  use 

Robert  lenders 

Arden  Syntax 

Mor  Peleg 

Contrasts  between  modeling  approaches  for  clinical  guidelines  and 
medical  decision  rules 

Vimla  Patel 

Cognitive  issues  in  guideline  representation  and  use 

Lucila  Ohno-Machado 

Description  of  breakouts 

The  second  plenary  period  was  split  into  two  sessions:  technical  and  non-technical 
Topics  presented  in  the  technical  plenary  session  were: 


Dan  Russler  and 

Gunther  Schadow 

Developing  Guideline  Messages  Using  the  Clinical  Classes  in  the  HL7 
Reference  Information  Model 

John  Fox 

Guardian  agents:  the  PROforma  approach  to  quality  and  safety  of 
interactive  guidelines 

Colin  Gordon 

PRESTIGE:  An  approach  to  shareable  guidelines  proven  in  6  EU 
countries 

Lucila  Ohno-Machado 

GLIF3 

Mario  Stefanelli 

From  Guidelines  to  Workflows 

Samson  Tu 

Toward  a  Task-Specific  Component-Based  Guideline  Modeling 
Architecture:  The  EON  Approach 

Topics  presented  in  the  non-technical  plenary  were: 


Christel  Mottur-Pilson 

Development  and  Dissemination  of  Guidelines 

Johan  van  der  Lei 

Garbage  in,  Garbage  out:  The  importance  of  well-designed  practice 
guidelines 

Jose  Arocha 

Interpreting  Guidelines  Using  Propositional  and  Semantic  Analyses: 
Implications  for  authoring  tools 

Alexa  McCray 

Developing  a  national  clinical  trials  registry 

William  Yasnoff 

Presenting  CDC  Guidelines  at  the  Point  of  Clinical  Decision 

Keith  Campbell 

Terminology:  one  size  does  not  currently  fit  all 
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There  were  five  breakout  sessions  that  consisted  of  initial  brief  presentations  followed  by  group 
discussions  on  focused  topics.  The  breakout  groups  reported  back  to  the  joint  session.  The 
following  is  the  summary  of  the  five  breakouts: 

Breakout  Topic:  Functional  Requirements 

Discussion  leader:  Perry  Miller,  Daniel  Kent 

Recorder:  Edward  Shortliffe,  Mor  Peleg 

Goals/Subtopics: 

Although  many  groups  have  individually  worked  on  defining  the  design  criteria  and  functional 
requirements  for  a  computer-based  representation  of  clinical-guidelines,  most  such  efforts  have 
been  guided  by  local,  application-specific  considerations.  In  this  breakout  session  we  will  try  to 
generalize  the  specification  of  such  criteria  and  requirements,  working  from  the  experiences  of 
the  individuals  involved  and  from  draft  documents  and  position  papers  that  have  proposed  the 
range  of  guiding  principles  for  a  guideline  representation  language.  Issues  for  discussion  will 
include: 

•  What  are  the  types,  applications,  and  use  environments  of  guidelines? 

•  What  are  the  key  elements  that  must  be  included  in  a  computer-based  representation  for 
guidelines? 

•  What  does  it  mean  for  an  encoded  guideline  to  be  computable? 

•  What  levels  of  abstraction  best  serve  the  process  of  authoring  and  understanding  clinical 
guidelines? 

•  How  should  ambiguities  be  managed? 

•  How  should  a  representation  deal  with  decisions  for  which  the  evidence  is  absent  or 
conflicting? 

•  What  kind  of  support  is  required  for  viewing  and  comprehending  the  logic  of  a  clinical 
guideline? 

•  How  should  a  representation  best  support  local  adaptation  and  implementation  of  a 
clinical  guideline? 


Breakout  Topic:  Model  and  Representation 
Discussion  leader:  John  Fox,  John  Gennari 
Recorder:  Aziz  A.  Boxwala,  Qing  Zeng 
Goals/Subtopics: 

The  discussions  in  this  breakout  session  will  focus  on  issues  of  knowledge  representation 
including  knowledge  modeling  and  representation  formats.  Topics  for  discussion  include: 

•  Paradigms  for  representation  of  guideline  knowledge  (such  rules,  fi'ames,  probabilistic 
methods,  etc) 

•  Formats  for  sharable  guidelines 
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•  Technical  issues  in  interfacing  with  complementary  standards  such  as  for  messaging 
(e.g.,HL7),  vocabularies  (e.g.,  SNOMED-RT) 

•  Technical  issues  in  integration  with  extant  knowledge  representation  formats  5.  Issues  in 
linking  to  electronic  medical  records  6.  Methods  for  assuring  quality  and  safety  of 
content 

Breakout  Topic:  Special  Issues  Relating  to  Clinical  Trials 
Discussion  leader:  Alexa  McCray,  Carol  Broverman 
Recorder:  Lucila  Ohno-Machado,  Samson  Tu 
Goals/Subtopics: 

Clinical  trial  CT  protocols  are  distinguished  guidelines  that  share  many  features  commonly 
found  in  other  clinical  guidelines.  However,  they  are  often  described  to  be  more  deterministic 
than  standard-of-care  guidelines.  This  distinction  is  due  to  the  nature  of  research  studies,  the 
trials  authoring  and  approval  process,  and  the  well-defined  and  regulated  data  gathering 
requirements  associated  with  clinical  trials.  Sponsors  and  the  conducting  sites  also  have  different 
views  of  the  protocol  that  must  be  supported.  The  representation  and  management  of  clinical 
trials  in  a  computerized  setting  must  also  accommodate  the  dissemination  of  updates, 
amendments  and  revisions  to  the  protocols. 

The  goal  of  this  session  is  to  focus  on  the  special  needs  to  support  clinical  trials  management, 
ranging  from  representation  requirements,  functional  requirements,  and  infrastructure 
requirements.  What  are  the  distinct  differences  between  clinical  trial  protocols  and  clinical 
guidelines? 

Topics  for  discussion: 

•  CT  protocols:  Overall  structure  and  requirements 

•  CT  procotol  modeling  and  workflow  management 

•  CT  eligibility  criteria 

•  CT  adverse  events  and  randomization 

•  CT  common  data  elements  (NCI) 

•  CT  site  and  sponsor  requirements 

•  CT  conditionality  and  decision-making 

•  CT  temporal  issues 
Breakout  Topicftnfrastructure  and  Tools 
Discussion  leader:  John  Silva,  Chip  Masarie 
Recorder:  Omolola  Ogrmyemi 
Goals/Subtopics: 

It  is  clear  that  guidelines  and  protocols,  used  during  patient  encoimters,  significantly  reduce  the 
risk  to  both  patient  and  physician.  It  is  also  abimdantly  clear  that,  unless  these  gmdelines  and 
protocols  are  transparently  used  within  the  encoimter,  most  physicians  don’t  or  won’t  use  them. 


7 


How  to  get  more  physicians  and  /  or  patients  to  use  tools  is  a  central  issue.  It  is  attractive  to 
imagine  that  guidelines  or  protocols  could,  some  day,  “self  install”  within  the  infrastructure  of 
the  physician’s  office  and  become  an  integral  component  of  care  delivery  processes. 

The  National  Cancer  Institute,  as  well  as  other  national-level  organizations,  have  an  even  greater 
challenge:  how  do  they  publish  a  text  document  that  specifies  a  practice  guideline  or  clinical  trial 
and  generate  the  computer-readable  specifications  that  would  permit  the  trial  to  be  installed  in 
ANY  authorized  place  of  practice?  This  includes  cancer  centers,  cooperative  groups,  smaller 
community  cancer  centers,  oncology  practices,  etc.  This  track  will  explore 

1 .  how  to  generate  computable  specifications  for  a  clinical  trial  [or  guideline], 

2.  what  tools  and  infrastructure  are  needed  on  the  generating  end 

3.  what  tools  and  infrastructure  are  needed  on  the  receiving  end 

4.  how  to  maintain  trials  content  between  ALL  the  sites  and  the  originator,  specially  for  updates, 
amendments,  revisions,  etc 

5.  what  is  the  architecture  needed 

6.  what  should/could  be  the  first  set  of  experiments 

Breakout  Topic:  Creation  of  a  process  for  standardization  of  a  guideline 
Discussion  leader:  Daniel  Kent,  John  Dulcey 
Recorder:  Robert  A.  Greenes 
Goals/Subtopics: 

To  continue  the  work  of  converging  various  initiatives  toward  a  standardized  approach  to 
representation  and  sharing  of  clinical  guidelines,  an  organizational  entity  needs  to  be  charged 
with  that  responsibility,  the  various  stakeholders  must  be  actively  involved,  key  individuals  must 
be  committed  to  this  activity,  and  the  process  must  have  a  means  for  financial  support  for  this 
continuing  work.  Issues  to  be  considered  during  this  breakout  session  include  but  are  not  limited 
to  the  following: 

•  What  organizational  form  is  most  appropriate,  e.g.,  independent  non-profit  consortium, 
alignment  with  or  subgroup  of  an  existing  SDO? 

•  What  activities  should  the  entity  take  on,  e.g.,  white  papers,  standards  documents, 
meeting  sponsorship,  open  source  software  tool  repository  and  distribution? 

•  What  kinds  of  membership  should  the  entity  have,  e.g.,  individual  vs.  institutional,  or 
both,  what  privileges  associated  with  each? 

•  What  are  means  for  supporting  the  entity's  ongoing  activities,  e.g,  grants,  membership 
fees  (possibly  tiered)? 

Key  Research  Accomplishments 

The  meeting  accomplished  its  primary  objective  of  initiating  a  dialog  on  issues  on  sharing  of 
computer-based  guidelines.  The  presentations  from  the  workshop  and  the  siunmaries  of  the 
breakout  sessions  are  attached  in  the  appendices.  These  may  also  be  accessed  from 
wvyw.glif.org. 
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Following  the  meeting,  e-mail  listserv  discussion  groups  were  organized.  Five  groups  were 
started,  one  for  each  of  the  breakout  sessions.  Information  on  how  to  subscribe  to  these  lists  is 
available  at  www.gliforg. 

Following  the  meeting  a  dialog  was  initiated  with  HL7  organization  to  examine  how  the  process 
of  standardization  of  a  format  for  shared  guidelines  should  be  accomplished.  In  September  2000, 
the  HL7  Board  approved  of  the  formation  of  a  Guideline  Interchange  Format  Special  Interest 
Group.  The  GLIF  SIG  is  part  of  the  Decision  Support  Technical  Committee  of  HL7.  The 
mission  of  this  SIG  is  “. . .  to  create  the  standard  electronic  representation  for  the  specification  of 
clinical  practice  guidelines  for  purpose  of  interchange  and  to  facilitate  their  integration  into  the 
healthcare  systems,  computer-based  medical  records,  and  a  variety  of  applications.” 

Reportable  Outcomes 

Key  outcomes  of  the  meeting  are 

1 .  Eighty  attendees  participated  in  a  2-day  workshop  aimed  at  fostering  sharing  of  computer- 
based  guidelines  and  developing  representations  for  shared  guidelines  (Appendix  A) 

2.  Thirty-one  position  statements  were  submitted  for  the  workshop  and  published  on  the 
Internet  at  wvyw.gliforg  (Appendix  B) 

3.  Each  breakout  group  created  a  summary  of  its  discussions  that  outlined  the  problems  to  be 
addressed,  priorities,  future  plans,  and  parties  that  would  be  appropriate  for  working  on  these 
issues  in  the  future.  These  summaries  are  published  on  the  Internet.  (Appendix  C) 

4.  The  functional  requirements  breakout  group  produced  a  fairly  comprehensive  set  of 
functional  requirements  for  a  shared  guideline  representation.  (Appendix  D) 

5.  Presentation  on  the  workshop  was  made  at  the  Annual  Meeting  of  the  American 
Telemedicine  Association  in  Phoenix,  Az  in  May,  2000. 

6.  E-mail  listservs  were  created  to  facilitate  continuing  discussions  on  these  topics. 

7.  Formation  of  the  GLIF  SIG  in  HL7  as  a  consequence  of  post-workshop  discussions  with  the 
HL7  leadership. 

Conclusions 

The  workshop  brought  together  several  of  the  key  stakeholders  to  address  issues  of  sharing 
clinical  guidelines  and  develop  representations  for  shared  guidelines.  There  was  a  strong 
consensus  among  participants  that  this  topic  is  important  and  that  work  on  it  should  continue. 
The  workshop  was  crucial  in  initiating  discussions  that  will  continue  in  other  forums  (electronic, 
HL7,  scientific  and  industry  conferences).  We  expect  that  these  discussions  will  produce  a 
process  and  standard  for  representation  of  shared  guidelines. 

References 
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Construction  of  a  Semantic  Meta-vocabulary 
on  Sharable  Guideline  Representation 

Knowledge  representation  has  always  been  a  challenge  on  the  development  of 
Health  Information  systems.  Despite  of  an  implicit  definition  on  a  sharable 
guideline  representation,  and  somewhat  explicit  on  the  structure  of  the  guideline 
model  of  the  GLIF  format,  some  questions  emerge  from  the  evaluation  of  the 
type  of  information  that  is  intended  to  be  shared  among  computer  medical 
systems. 

As  mentioned  in  some  studies,  the  translation  of  unstructured  data  from  existing 
guidelines  into  structured  representation  format  imposes  an  extra  vigor  and  an 
inconvenient  jeopardy  to  data  quality.  Also,  most  of  cognitive-evaluation 
techniques  are  incipient  tasks  while  dealing  with  natural  language  processing. 

The  absence  of  an  altered  univocal  procedure  between  a  text-based  guideline 
and  GILF  format  (one  text  guideline  can  generate  several  GLIF-based 
guidelines)  arises  an  insufficient  semantic  scope  about  the  quality  of  the  shared 
information. 

Expressing  a  structured  guideline  without  ambiguity  forces  a  construction  of  a 
semantic  meta-information  vocabulary  over  the  information  acquired  during  the 
translation  process.  This  new  vocabulary  should  describe  the  wholeness  of  the 
expected  information  by  the  system  and  also  information  originated  throughout 
this  process,  including  support  for  general  and  international  measurement 
conventions  systems  (i.e.,  weight,  height,  temperature,  etc.),  medical  codification 
(ICD-9,  ICD-10,  SNOMED,  etc.)  and  different  language  endorsement  (i.e., 
Portuguese,  French,  Spanish,  etc.). 

For  instance,  an  unstructured  data  (i.e.  fever)  translated  into  structured  format 
(i.e.,  temperature  >  xx)  should  be  notarize  from  this  meta-vocabulary  (i.e., 
Celsius  or  Fahrenheit)  to  strengthen  the  comprehension  of  the  information  and 
enrich  the  uniqueness  of  encoding  process. 

Nevertheless,  the  problem  of  relying  subjectively  on  the  encoder  knowledge  will 
persist,  as  semantic  reasoning  depends  on  medical  background  knowledge. 

Furthermore,  a  special  attention  should  be  rendered  to  the  development  of  on- 
the-fly  translators  systems  to  shorten  the  processing  path  and  minimize  the 
conflict  between  formal  data  interpretation  and  subjective  meanings  of  guideline 
representation. 
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Federal  University  of  Sao  Paulo,  Brazil 
Rua  Botucatu  862,  Ed.  Leal  Prado 
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Carol  A.  Broverman,  Ph.D  . 

Michael  G.  Kahn,  M.D.,  Ph.D. 

Fast  Track  Systems 
San  Mateo,  California 
cbroverm@sprynet.com 

One  of  the  primary  obstacles  in  establishing  widespread  acceptance  of  any  decision  support  system  is  the 
difficulty  in  integrating  such  a  system  into  the  patient  care  workflow  and  existing  information  systems. 
Deployments  of  such  systems  always  require  extensive  work  efforts  to  build  institution  or  database-specific 
conduits  to  deal  appropriately  with  local  schemas,  data  types,  and  vocabulary  terms.  The  effort  required  to 
integrate  a  new  system  into  an  existing  infrastructure  represents  both  a  significant  start-up  and  maintenance 
resource  cost  to  the  institution;  and  a  nearly  insurmountable  barrier  to  technology  sharing  across 
institutions,  technologies,  and  vendors. 

The  work  to  date  on  knowledge-based  guideline  and  rule  standards  (specifically  GLIF  and  the  Arden 
Syntax)  has  concentrated  on  the  modeling  of  clinical  rule  and  guideline  logic  from  a  process  and 
knowledge  model  perspective.  A  critical  piece  of  the  puzzle  that  has  understandably  been  sidestepped  is  the 
integration  of  patient-centered  data  elements  and  the  accompanying  “terminology  integration.”  A  typical 
approach  taken  within  guidelines  developed  by  the  collaborators  is  the  use  of  predicates  and  other 
constructs  to  reference  clinical  concepts  within  a  locally-defined  ontology,  a  solution  which  is  not 
shareable.  This  missing  piece  is  what  has  been  referred  to  as  the  infamous  “curly  braces”  problem  within 
the  Arden  Syntax  community  [1].  The  reason  for  this  position  has  been  the  lack  of  an  agreed-upon 
information  model  for  patient  information  and  clinical  concepts,  and  a  lack  of  a  unified  medical 
terminology  by  which  specific  clinical  concepts  can  be  uniquely  identified. 

It  is  our  position  that  solutions  to  this  issue  should  leverage  ongoing  work  within  other  standards  bodies 
that  include  representation  from  all  of  the  following  areas:  academic  medical  centers,  govermnent 
sponsored  work  groups,  and  the  industry  sector.  Specifically,  we  belief  that  Health  Level  Seven  (HL7),  an 
ANSI  approved  standards  body,  has  become  a  locus  of  activity  that  brings  together  these  different  interest 
groups.  Within  this  framework,  we  suggest  the  consideration  of  a  strategy  to  devise  data  query  constructs 
in  the  guideline  model  that  is  consistent  with  the  Reference  Information  Model  (RIM),  the  HL7  Version  3.0 
data  types,  and  the  “value  domain  specifications”  currently  being  devised  within  various  HL7  committees. 

Previous  work  reports  on  the  use  of  the  HL7  Reference  Information  Model  to  address  the  infamous  “curly 
braces”  problem  within  the  Arden  Syntax  [2];  a  problem  which  persists  within  the  usage  of  GLIF  and  other 
related  formalisms  in  the  various  institutions  of  the  InterMED  collaboratory  today.  We  believe  that  since 
this  initial  report,  the  work  on  the  RIM  has  xmdergone  numerous  iterations  and  harmonizations,  and  has 
evolved  into  a  more  broad-based  work  that  also  incorporates  constructs  to  support  guidelines.  This  work 
represents  the  collaborative  work  of  the  Patient  Care  Technical  Committee  and  the  Orders  and  Results 
Committee,  and  has  culminated  in  the  incorporation  of  the  resultant  Unified  Service  Action  Model  (USAM) 
into  the  RIM  [3]. 

Furthermore,  it  is  possible  to  specify  data  type  and  terminology  constraints  within  guidelines  and  protocols 
by  using  the  data  types  and  syntactic  terminology  domain  specifications  being  developed  by  the 
Control/Query  and  Vocabulary  Technical  Committees  within  HL7  [4].  For  example,  standardized  qualifiers 
such  as  domain  name  (e.g.;Clinical  Diagnosis),  realm  (e.g.;  USA),  and  code  system  (e.g.;  SMI  for 
SNOMED  International)  as  catalogued  in  a  Value  Set  Definition  Table  could  be  referenced  within  a 
guideline.  These  qualifiers,  along  with  a  particular  clinical  expression  could  be  passed  to  a  terminology 
server  to  resolve. 


We  also  advocate  that  the  information  representations  and  data  dictionaries  being  developed  by  other 
related  organizations  be  registered  in  a  central  repository  with  other  terminologies  such  as  SNOMED-RT, 
MedDRA,  ICD-9-CM,  LOINC,  and  others,  and  harmonized/mapped  where  possible.  Examples  of  such 
other  related  efforts  are  the  Common  Data  Elements  (CDE’s)  being  produced  by  the  Cancer  Information 
Infrastructure  (CII)  of  the  National  Cancer  Institute  (NCI)  [5],  and  the  glossary,  meta-model  and  other 
workproducts  being  produced  by  the  Clinical  Data  Interchange  Standards  Committee  (CDISC  ~  a  working 
group  within  the  Drug  Information  Association  (DIA)  [6]).  For  example,  it  might  be  possible  to  fold  in  the 
work  on  Common  Data  Elements  within  the  CII/NCI  into  a  more  rigorous  data  model  that  could  be  a  subset 
of  vocabulary  “registered”  within  the  HL7  framework. 

Other  possible  areas  for  collaboration  and  advance  would  involve  standardizing  the  language  of  eligibility 
criteria.  This  language  would  create  associations  to  a  centralized  bank  of  terminologies,  using  the 
standardized  syntactic  expressions  for  terminology  domains.  This  proposed  project  would  bring  together 
related  on-going  work  at  universities,  NCI,  NLM,  CDC  and  FDA.  The  involvement  of  industry  would  be 
needed  to  appropriately  tech-transfer  this  work  and  bring  about  a  wider  deployment  of  a  “terminology- 
enabled”  guideline  and  protocol  standard. 

[1]  Hripcsak  G,  Johnson  SB,  Clayton  PD.  Desperately  seeking  data:  knowledge  base-database  links. 
Proceedings  of  the  Annual  Symposium  of  Computing  Applications  in  Medical  Care  1993;:639-643. 

[2]  lenders  RA,  Sujansky  W,  Broverman  CA,  Chadwick  M.  “Towards  improved  knowledge  sharing: 
assessment  of  the  HL7  Reference  Information  Model  to  support  medical  logic  module  queries, 
“Proceedings  of  the  AMIA  Annual  Fall  Symposium,  1997;:308-12  . 

[3]  Russler  DC,  Schadow  G,  Mead  C,  Snyder  T,  Quade  LM,  McDonald  CJ.  Influences  of  the  unified 
service  action  model  on  the  HL7  reference  information  model.  Proceedings  of  AMIA  Symposium  I999;(l- 
2):930-4. 

[4]  www.hl7.org  (see  work  of  Vocabulary  and  Control/Query  subcommittees) 


[5]  cii.nci.nih.gov/cde 

[6]  www.diahome.org/cdisc 
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Introduction 

The  development  of  traditional,  paper-based  methods  for  disseminating  guidance 
on  best  clinical  practice  has  been  accompanied  by  the  development  of  accepted  processes 
for  ensuring  the  quality  of  the  information  published.  The  current  development  of  new 
paradigms  for  the  electronic  dissemination  and  consultation  of  clinical  material  - 
including  enactable  clinical  guidelines  and  decision  support  systems  -  must  also  be 
accompanied  by  the  development  of  appropriate  new  methods  for  ensuring  the  quality 
and  safety  of  the  resultant  systems. 

A  distinction  should  be  drawn  between  the  clinical  validity  of  the  content  of  a 
decision  support  application,  and  technical  aspects  of  the  representation  and  delivery  of 
that  content.  A  unifying  methodology  for  quality  assurance  will  need  to  embrace 
approaches  from  at  least  4  different  areas: 

•  documentation  standards  to  support  clinical  quality  and  accountability 

•  a  rigorous  publishing  and  evaluation  cycle  supporting  the  medical  content  and  its  evidence  base 

•  software  engineering  methods  to  support  application  integrity  /  reliability 

•  the  development  of  software  capable  of  supporting  active  safety  management  and  dynamic 

management  of  unforeseen  hazards 

Mechanisms  for  ensuring  the  quality  of  the  clinical  knowledge  base 

One  approach  to  quality  assurance  is  the  use  of  documentation  fields,  such  as  the 
“Maintenance”  and  “Library”  slots  used  in  the  Arden  Syntax  Medical  Logic  Modules. 

We  propose  to  extend  this  approach  to  differentiate  between  clinical  concerns,  such  as 
validity,  applicability,  and  authorship,  and  technical  concerns  such  as  version  control  and 
interoperability. 

We  believe  that  the  use  of  such  documentation  fields  also  needs  to  be 
complemented  by  a  structured  development  and  reappraisal  process,  paralleling  that  used 
by  the  pharmaceutical  industry  in  evaluating  new  drugs.  The  validation  of  an  application 
on  test  cases  (“paper  patients”)  might  be  the  first  step  in  this  process,  but  would  represent 
only  the  start  of  a  continuing  validation  and  refinement  cycle.  In  such  a  cycle,  data  on 
the  impact  of  guideline  use  on  clinical  outcomes  would  be  fed  back  systematically  into 
the  authoring  process.  Use  of  Electronic  Patient  Record  and  statistical  analysis  and  data 
mining  techniques  will  facilitate  the  discovery  of  more  complex  relationships  between 
patient  presentations,  clinical  interventions  and  clinical  outcomes  than  is  feasible  at 
present.  In  effect,  every  patient  treated  according  to  an  electronic  clinical  guideline  is 
automatically  entered  into  an  ongoing  clinical  trial  of  that  guideline,  with  immediate 
potential  to  collect  and  analyse  data  in  real  time. 


Mechanisms  for  ensuring  the  safety  and  reliability  of  the  delivery  technology 

The  efficacy  of  individual  applications  cannot  be  considered  in  isolation;  in  our 
view  we  also  have  an  obligation  to  show,  as  far  as  possible,  that  the  general  techniques 
are  formally  sound  and  that  the  technology  is  safe  (Fox  and  Das,  forthcoming).  There  has 
been  grov^ng  interest  in  applying  techniques  from  formal  software  engineering  to 
decision  support  systems  in  recent  years.  The  motivation  for  developing  formal  design 
techniques  has  been  both  the  desire  to  remove  many  of  the  apparently  ad  hoc  practices 
associated  with  decision  support  systems  development,  and  to  provide  techniques  for 
automated  verification  and  validation  of  the  system  knowledge  base. 

We  have  also  argued  that  in  complex  domains  like  medicine  even  the  most 
rigorous  design,  validation  and  verification  will  not  guarantee  that  clinical  misadventures 
will  not  occur  in  practice.  It  is  humanly  impossible  for  software  designers  to  anticipate  all 
the  hazards  that  can  arise  due  to  unforeseen  and  unforeseeable  interactions  that  may 
occur  in  the  pressures  of  routine  clinical  medicine.  We  therefore  believe  that  medical 
informaticians  must  address  dynamic  as  well  as  static  aspects  of  the  system  design.  The 
approach  that  we  are  adopting  is  to  incorporate  active  hazard  management  systems  into 
the  guideline  delivery  technology.  These  “guardian  agents”  are  intelligent  systems  in 
their  own  right  but  their  expertise,  the  detection  and  management  of  hazards,  is 
complementary  to  the  medical  knowledge  which  is  encoded  in  the  guideline  itself  (Fox 
and  Das,  op  cif). 


Application  development  stage 

Application  deployment  stage 

Clinical 

content 

Medical  documentation  including : 

•  Content  authorship 

•  Degree  of  validation 

•  Links  to  further  information 

•  Test  cases  used  during 
validation,  or  other  evidence 
supporting  validation  status 

Structured  lifecycle  methodology, 
supporting  the  feedback  of  clinical 
outcomes  into  content  revision. 

Technical 

platform 

Software  design : 

•  CASE  tools  for  authoring, 
checking  and  testing 

•  Use  of  a  formally  grounded 
interchange  format 

Technical  documentation  including : 

•  Guideline  logic 

•  Design  rationale 

Support  for  software  methods 
capable  of  actively  anticipating 
problems,  and  responding  to 
unforeseen  hazards. 

Table  :  Key  features  of  a  proposed  unified  quality  and  safety  methodology,  providing  support  for 
clinical  and  technical  quality  assurance,  during  the  development  of  applications  and  during  their 
operational  lifecycle. 
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Introduction 

With  the  increased  emphasis  in  applying  evidence  based  medicine  to  everyday  care,  there 
has  also  been  a  renewed  interest  in  clinical  practice  guidelines.  Computerized  guidelines 
have  been  shown  as  an  effective  means  of  improving  guideline  compliance*'^.  While 
developing  a  guideline  engine  for  a  diabetes  operations  improvement,  we  decided  upon  a 
representation  based  on  the  GLIF  -  the  GuideLine  Interchange  Format^.  Others  have  also 
studied  and  used  GLIF  as  well*^'^.  We  chose  this  formalism  because  it  can  be  easily  (and 
exactly)  represented  in  a  flowchart,  a  representation  familiar  to  clinicians.  This  was  a 
significant  advantage  in  developing  the  guideline  content  with  computer-naive  clinicians. 

While  building  this  system,  we  encountered  limitations  in  the  GLIF  specification.  These 
limitations,  while  not  so  apparent  in  the  static  version  of  guideline  content,  became 
readily  so  in  the  context  of  real  world  implementation.  These  limitations  can  be  described 
as  being  in  one  of  four  categories;  incomplete  and  missing  data,  interactive  versus  batch 
(or  daemon)  mode  execution,  mapping  of  data  elements  in  guideline  steps  to  outside  data 
sources,  and  structuring  of  guideline  output.  In  this  position  statement,  we  will  describe 
each  of  these  categories  in  more  detail,  and  briefly  mention  how  we  addressed  them. 

Incomplete  and  missing  data 

Data  availability  is  a  significant  factor  in  implementing  computerized  guidelines.  In  the  context  of 
the  diabetes  operations  improvement  project,  it  became  immediately  apparent  that  missing  data 
items  would  be  a  common  occurrence.  In  thinking  about  missing  data  and  its  impact  on  specific 
guidelines,  we  made  2  observations.  First,  that  not  all  data  items  have  the  same  importance;  some 
data  elements  are  more  significant  than  others.  While  some  missing  data  elements  might  cause  a 
guideline’s  execution  to  halt,  others  may  not.  The  second  observation  we  made  was  that,  in 
general,  the  more  information  you  have,  the  more  specific  advice  you  can  derive  from  a  practice 
guideline.  Thus,  even  if  the  execution  of  a  guideline  did  stop,  some  advice  (albeit  more  general) 
could  still  be  given.  To  incorporate  these  observations  into  our  implementation,  we  did  the 
following:  first,  we  used  a  tri-value  logic  scheme  (true/false/unknown).  Situations  where  not  all 
data  elements  were  known  would  result  in  an  “unknown”  value.  Second,  we  extended  the  logic  of 
the  guideline’s  execution  model  such  that  it  would  this  handle  the  third  “unknown”  value  properly. 
Third,  we  incorporated  within  the  framework  of  our  implementation  the  ability  to  “remember”  all 
unfulfilled  data  requests,  such  that  they  could  be  returned  to  the  client  app  (along  with  output 
advice). 

Interactive  versus  batch  (or  daemon)  mode  execution 


Computerized  guidelines  can  be  executed  in  one  of  two  modes:  in  an  “interactive”  mode,  where 
there  is  a  human  user  that  can  fill  in  missing  items  (i.e.  a  guideline  system  vrithin  an  order  entry 
system),  or  in  a  “batch”  mode,  where  the  guideline  is  running  as  a  background  process  (i.e.  a 
guideline  system  that  prints  reminders  on  a  summary  page  prior  to  a  clinic  visit).  In  the  latter 
situation  there  is  no  human  user  to  supply  missing  data.  A  sharable  guideline  formalism  should  be 
able  to  handle  both  contingencies. 

In  our  implementation,  we  treated  GLIF  steps  as  ftmctions  that  take  as  input  and  return  as  output  a 
“data  tree  object.”  This  data  tree  object  represents  the  “state”  of  the  guideline  engine  at  any  given 
point.  As  guideline  execution  progresses  from  GLIF  step  to  GLIF  step,  this  “data  tree  object”  is 
chained  from  one  GLIF  step  to  another.  Because  this  “data  tree  object”  stores  the  ‘state”  of  the 
guideline  system,  all  that  would  be  required  to  change  the  mode  of  execution  from  batch-mode  to 
interactive  mode  would  be  to  pass  this  object  to  a  user  interface  before  it  goes  to  the  next  GLIF 
step. 

Mapping  of  data  elements  in  guideline  steps  to  outside  data  sources 

This  is  a  significant  problem;  in  one  experiment  involving  sharing  of  Arden  Syntax  encoded 
MLM’s,  most  had  to  be  modified;  the  most  frequently  modified  area  was  the  “data”  section^®.  This 
data  mapping  problem  in  MLM’s  is  often  referred  to  as  the  “curly  braces”  problem  (because  the 
data  section  in  Arden  Syntax  is  denoted  by  curly  braces).  Ideally,  the  implementation-specific 
details  of  data  acquisition  should  be  encapsulated  from  the  logic  specification  of  the  guideline.  The 
initial  version  of  GLIF  provides  no  systematic  formalism  for  representing  these  data-mapping 
issues.  In  our  implementation,  we  encapsulated  the  “logic”  portion  of  the  guideline  engine  from 
the  “data  acquisition”  part  by  using  an  XML-based  interface.  We  chose  XML  because  at  the  time  it 
was  an  emergent  web-based  standard;  in  theory  any  scheme  capable  of  supporting  hierarchical  data 
structures  would  suffice. 

Structuring  of  guideline  output 

We  came  to  the  conclusion  that  the  output  from  a  guideline  engine  should  be  structured.  We 
arrived  at  this  conclusion  from  multiple  perspectives.  First,  within  the  context  of  the  diabetes 
project,  we  needed  to  have  2  views  of  the  guideline  recommendations:  a  “headline”  view,  and  a 
more  detailed  “drill-down”  view.  Second,  from  examining  the  GLIF  standard,  it  became  apparent 
that  in  order  for  GLIF  to  truly  support  nested  subguidelines,  there  would  have  to  be  some  standard 
method  to  access  the  results  of  a  given  subguideline.  Third,  structure  would  be  important  to  make 
the  output  computer-friendly  to  other  informatics  applications,  such  as  order-entry  systems  or 
prescription  refill-printing  applications.  The  “data  tree  object”  from  our  guideline  engine 
implementation  allowed  for  the  ability  for  recommendations  to  be  inserted  in  a  ordered  fashion. 
Because  the  inherent  structure  of  the  “data  tree  object”  is  hierarchical,  mapping  to  and  from  a  “data 
tree  object”  and  an  XML  serialization  is  trivial.  The  structure  used  in  our  implementation  is 
simplistic  and  was  obtained  in  an  ad-hoc  fashion;  since  then,  we  have  begun  work  on  developing  a 
more  formal  model  of  guideline  output”. 
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Guideline  Integration  Requires  an  Underlying  Common 
Representational  Schema 


By:  Peter  L.  Elkin,  MD 

Statement  of  the  Problem 

Interoperability  is  required  to  make  logical  and  reasonable  decisions  regarding 
inferencing  across  guidelines.  Sharing  guidelines  can  be  conceptualized  in  two  distinct 
ways.  First,  there  is  the  issue  of  communicating  guidelines  from  institution  to  institution. 
Second,  there  is  the  issue  of  communicating  across  guidelines  when  situations  arise 
where  individual  patients  may  be  candidates  for  the  application  of  multiple  guidelines  or 
portions  of  those  guidelines.  Both  of  these  cases  require  interoperability. 

In  the  first  case,  it  is  clear  that  many  institutions  have  created  their  own  guidelines. 
These  by  definition  vary  in  complexity  and  in  the  language  used  to  represent  the  decision 
points  and  actions  described  by  the  guideline  (and  perhaps  even  the  conditions  to  which 
the  guideline  is  intended  to  be  applied).  In  the  second  case  one  needs  to  identify  the 
common  nodes  across  guidelines,  which  can  be  used  as  bridges  to  other  guidelines  which 
may  apply  to  certain  patient  conditions.  For  example,  a  guideline  for  the  treatment  of 
“Congestive  Heart  Failure”  (chf)  might  very  well  recommend  the  institution  of  a  Beta 
Blocker.  However,  the  patient  may  develop  symptomatic  bradycardia  from  the 
medication  and  the  former  guideline  could  link  to  a  guideline  for  the  treatment  of 
symptomatic  bradydysrhythmias.  In  another  example,  the  same  patient  might  be  advised 
by  the  protocol  to  be  placed  on  Digoxin,  but  the  protocol  notes  that  this  is  risky  if  the 
patient  is  hypokalemic,  the  chf  guideline  might  link  to  another  guideline,  which  addresses 
the  treatment  of  hypokalemia.  Links  such  as  these  are  based  on  the  availability  of  a 
robust  underlying  common  representational  schema. 

Over  the  last  several  years  there  has  been  considerable  advancement  within  the  field  of 
controlled  health  vocabularies.  Large-scale  terminologies  have  been  developed  which 
have  formal  as  well  as  systematic  definitions  associated  ’with  them.  These  formalisms  are 
the  only  tractable  answer  to  several  problems  that  have  long  plagued  the  Informatics 
community. 

First,  we  need  to  be  able  to  recognize  redundancy  in  these  large-scale  terminological 
efforts  (often  they  contain  hundreds  of  thousands  of  terms).  When  dealing  with 
compositional  terminologies  (which  represent  the  set  of  the  most  robust  and  flexible 
representational  schemes),  one  must  be  able  to  normalize  the  terminology  in  order  to 
accomplish  the  task  of  eliminating  or  preventing  unrecognized  redundancy. 
Normdization  is  the  process,  which  anneals  duplicate  compositional  expressions  that 
have  the  same  meaning.  For  example,  if  the  term  “Cellulitis”  can  also  be  expressed  as  a 
compositional  expression  containing  “Inflammation”  with  Topology  “Skin”  vdth 
Etiology  “Infectious  Disease”  then  the  terminological  system,  if  normalized,  would 
recognize  this  identity. 


Second,  we  need  to  have  consistent  identification  of  appropriate  subsumptive 
relationships  vrithin  terminological  hierarchies.  This  is  insured  by  complete  description 
logic  based  terminologies,  as  these  relationships  are  generated  algorithmically  from  the 
description  logic  itself. 

Third,  we  need  to  be  able  to  map  between  different  representational  systems  as  “one  size 
does  not  fit  all.”  Here  we  begin  to  see  the  utility  of  these  terminological  systems  as  a 
tool  for  accomplishing  interoperability  between  guidelines  and  their  implementation. 
Each  of  the  rubrics  used  within  guidelines  produced  anywhere  in  the  world  could  be 
made  interoperable  by  mapping  them  to  a  common  underlying  representational  scheme. 
This  ability  is  the  cornerstone  of  the  terminological  support  needed  for  guideline 
integration. 


Proposed  Solution 

Indexing  each  of  the  guidelines  which  one  wishes  to  integrate  will  provide  a  mechanism  for 
interoperability.  This  includes  national  guidelines  needing  integration  with  local  or  regional 
guidelines  (intra-guideline  integration)  and  integration  between  guidelines  (inter-guideline 
integration).  This  facility  can  and  should  serve  as  the  portal  through  which  one  can  visualize  the 
similarities  between  and  among  guidelines. 

I  believe  that  this  in  and  of  itself  will  be  useful  in  gaining  local  and  regional  acceptance  of  national 
guidelines.  This  is  predicated  on  the  notion  that  one  of  the  principle  barriers  to  guideline 
acceptance  is  the  difference  in  common  parlance  which  exists  across  our  great  land.  This  can  be 
tolerated  and  indeed  encouraged  by  considering  these  differences  as  colloquial  terminologies  that 
can  be  mapped  neatly  to  a  consistent  underlying  reference  terminology. 

Today’s  current  guidelines  themselves  are  a  simplification  of  the  problem  of  consistent 
patient  management.  Today’s  robust  controlled  terminologies  are  still  a  form  of 
aggregation  and  cannot  and  do  not  attempt  to  code  all  aspects  of  patient  care.  That  said, 
there  is  a  considerable  expressivity  in  the  compositional  terminologies,  which  exist  today, 
and  this,  in  my  opinion,  is  a  small  price  to  pay  for  the  interoperability  we  seek  for  tasks 
such  as  guideline  integration. 

A  common  imderlying  representational  schema  is  essential  for  the  interoperability 
necessary  to  successfully  integrate  guidelines.  I  believe  that  this  requires  a  robust 
imderlying  terminological  system,  which  is  based  on  a  formal  definitional  schema.  I 
believe  that  such  a  mechanism  can  facilitate  guidelines  by  several  mechanisms.  First, 
these  mappings  can  serve  as  a  portal  between  guidelines  regarding  the  same  topic. 

Second  they  can  serve  as  a  portal  between  distinct  but  clinically  related  guidelines.  Third 
they  can  be  a  bridge  to  reference  information  to  justify  the  advice  given  in  the  guideline. 

Fourth  these  mappings  can  serve  as  a  mechanism  to  facilitate  diversity  in  representational 
form  without  sacrificing  clarity  of  meaning.  For  these  reasons  and  many  others,  which 
for  sake  of  space  limitations  were  omitted,  I  submit  that  the  underlying  mechanism  for 
the  interoperability  needed  to  accomplish  the  goal  of  guideline  integration  rests  squarely 
on  the  shoulder  of  today’s  and  future  controlled  health  vocabularies. 


Representations  for  sharable  guidelines:  convergence  or  diversity? 


John  Fox  and  Jonathan  Bury 
ICRF,  London,  28/1/2000 

As  the  participants  in  this  workshop  will  be  aware  the  current  massive  interest  in  the  creation  and  use  of  clinical 
practice  guidelines  is  something  of  a  mixed  blessing.  The  recognition  by  healthcare  professionals  that  much 
variability  in  quality  of  care  is  avoidable,  and  that  it  can  be  addressed  in  part  by  the  use  of  guidelines,  are 
significant  advances  of  course.  On  the  other  hand  the  value  of  the  countless  textTITML  documents  on  the  web 
which  purport  to  represent  “best  practice”  is  undermined  by  variability  in  their  quality,  in  both  presentation  and 
content. 

As  we  try  to  develop  executable  or  enactable  guidelines,  which  must  be  expressed  in  a  technical  language  such 
as  the  Arden  Syntax,  GLIF  or  PRO/orma,  formal  design  and  rigorous  development  become  even  more  critical.  A 
guideline  interchange  format,  like  any  other  computer  language,  must  provide  clear  standards;  agreed  terms  for 
representing  medical  concepts  and  an  unambiguous  notation  for  representing  clinical  procedures,  for  example.  If 
we  are  to  achieve  sharability  of  enactable  guidelines,  particularly  if  we  seek  reusability,  interoperability  etc.  then 
even  stricter  disciplines  must  be  observed. 

The  need  to  address  this  problem  was  first  recognised  in  the  MLM  model  of  the  Arden 
Syntax.  Although  experience  with  Arden  has  not  lived  up  to  all  the  designers’  hopes  the 
benefits  of  the  standards  it  introduced  were  considerable  and  it  has  been  widely  adopted.  The 
reasons  that  Arden  did  not  achieve  all  its  objectives  seem  similar  to  the  reasons  that  classical 
programming  paradigm  ran  into  trouble  20  years  ago;  they  did  not  “scale  up”.  In  due  course 
they  were  supplanted  by  modem  software  engineering  paradigms  like  object-oriented 
programming  and  the  use  of  CASE  tools  to  support  development  throughout  the  development 
life-cycle. 

In  the  last  few  years  we  have  seen  a  similar  trend  in  the  domain  of  enactable  guidelines.  There 
have  been  a  number  of  promising  proposals  for  guideline-oriented  representation  languages, 
interchange  formats,  protocol  execution  engines  etc.  Most  of  these  follow  a  paradigm  that  has 
several  distinctive  features.  Roughly  speaking  this  paradigm  emphasises  task-oriented 
representations  (rather  than  rule-based  reasoning  or  procedural  code),  declarative  description 
formats  (which  facilitate  automatic  checking  and  verification  better  than  traditional 
procedural  languages),  component-based  development  (for  reusability  and  reliability)  and 
lifecycle-based  CASE  tools  to  support  quality  development  and  maintenance  (e.g.  graphical 
authoring  tools).  Examples  of  guideline  technologies  that  embody  part  or  all  of  this  paradigm 
are  Shahar’s  ASBRU;  Musen  and  colleagues’  EON/Protege;  InterMed’s  GLIF  and  PRO/brma 
(see  figure).  These  technical  similarities  are  not  emerging  by  mere  coincidence  but  because 
they  embody  techniques  that  have  been  widely  and  successfully  used  in  conventional  software 
engineering,  and  more  recently  in  knowledge  engineering. 


View  of  a  guideline  authoring  system  for  the  PROforma  interchange  format  (one  of  the  Arezzo®  suite  of  CASE 
tools  developed  with  /nferMed  Ltd.)  The  central  drawing  area  allows  an  application  developer  to  lay  out  the  task 
structure  of  the  guideline,  here  a  National  Health  Service  cancer  referral  guideline,  using  standard  task  models 
for  decisions,  care  plans,  actions  and  data  enquiries  represented  as  simple  shapes.  The  thumbnail  view  on  the 
left  shows  a  hierarchical  view  of  the  guideline,  which  is  recursively  defined  over  subplans,  and  the  panel  on  the 
right  shows  the  declarative  task  specification  in  the  PROfo/ma  interchange  format. 


An  important  side-effect  of  this  growing  paradigm  is  that  it  creates  an  opportunity  for  different  research 
groups  to  converge  technically  and  this  should  promote  sharability  -  of  ontologies,  guideline  components  etc. 
In  the  case  of  PRO/bma  and  GLIF,  for  example,  there  appear  to  be  significant  similarities  between  some  of 
the  task  models  used  in  the  two  languages.  It  may  in  fact  be  possible  to  ensure  that  for  certain  classes  of 
guideline  the  PKOforma  definition  could  be  automatically  translated  into  GLIF,  and  vice  versa,  which  would 
permit  applications  developed  for  one  enactment  engine  to  be  reused  in  the  other.  We  might  even  agree  a 
standard  syntax  and  operational  semantics  for  the  common  task  types. 

Extrapolating  fi'om  this  observation,  one  might  argue  that  the  goal  of  the  continuing  collaborations  between 
members  of  the  “guidelines  community”  is  to  converge  totally,  and  so  we  should  agree  a  single  standard 
language.  Eventually  this  will  be  possible  and  desirable  but  we  anticipate  that  most  members  of  the 
commimity  will  feel  that  this  would  be  premature  now.  At  this  point  in  the  development  of  guideline 
technology  we  do  not  know  enough  about  the  technical,  clinical  or  human-interface  problems  in  om  area,  let 
alone  the  most  promising  solutions.  We  should  clearly  remain  alert  to  convergences  where  these  emerge,  and 
translate  them  into  common  standards  as  and  when  appropriate,  but  for  now  different  groups  are,  and  should 
be,  pmsuing  different  approaches. 


Representations  for  Sharable  Guidelines  :  Position  statement  for  the 

Boston  Meeting. 

Colin  Gordon  c.gordon@rbh.nthanies.nhs.uk 
[For  some  background  see:  smi- 
web.stanford.edu/projects/kmg/guideIine_panel/index.html  ] 

Development  and  dissemination  of  guidelines.  Guideline  authors  have  followed  the 
example  of  EBM  by  establishing  explicit  methodologies  for  guideline  development.  So 
far  however  little  has  been  done  to  render  explicit  the  processes  of  deriving 
recommendations  from  evidence,  or  the  ways  in  which  it  is  expected  that  guidelines  are 
to  be  used.  Many  guideline  authors  probably  envisage  their  use  as  a  generic  educational 
background  resource,  rather  than  for  case-by  case  consultation,  and  their  documents  are 
formulated  accordingly.  Some  guidelines  (e.g.  those  produced  by  SIGN  in  Scotland)  are 
explicitly  declared  not  to  be  intended  for  direct  use  in  practice  but  only  as  a  basis  for 
deriving  local  guidelines  where  issues  of  cost-effectiveness  and  resource  availability  will 
be  taken  into  account.  The  methodology  for  producing  locally  adapted  guideline  is  even 
less  well  defined  than  that  for  the  national  materials  they  use  as  their  basis.  It  would 
probably  be  salutory  for  all  concerned  if  guideline  authors  were  to  be  more  actively 
involved  in  the  design  and  implementation  of  procedures  for  disseminating  and  applying 
their  recommendations. 

Integration  of  guidelines  into  practice.  There  is  clear  evidence  that  guideline 
implementation  can  improve  practice,  especially  when  delivered  through  patient-specifie 
prompts.  Computer  systems  linked  to  an  EHR  (and  to  other  key  serviees,  notably  for 
prescribing  and  orders/communications)  have  proved  in  several  setting  effective  tools  for 
achieving  this  result.  This  does  not  mean  that  decision  support  in  the  consultation  is  the 
only  effective  way  to  communicate  guideline  knowledge.  Efficient  web-based  resources 
for  educational  and  reference  consultation  of  guidelines,  evidence  and  literature,  in 
various  hours  and  locations,  are  likely  to  remain  equally  indispensable  to  maintaining  the 
quality  of  healthcare  delivery.  Standard  mark-up  for  search  and  browsing  is  not  less 
important  than  standard  KR  for  decision  support. 

Knowledge-representation  and  functional  requirements  of  a  shared  guideline 
format.  Several  benefits  are  possible  from  common  approaches  and  convergence 
towards  standards.  Many  to  many  sideline  dissemination.  Guideline  authors  will  benefit 
from  a  KR  standard  which  makes  their  material  usable  in  multiple  sites  on  multiple 
technology  platforms.  The  benefit  increases  in  the  case  of  guidelines  designed  for 
cooperating  use  by  multiple  care  agents.  Vendors  and  users  of  clinical  informatics 
platforms  have  greater  incentives  for  installing  guideline  implementation  capabilities  if 
high  quality  guideline  material  is  available  from  several  reputable  authors  in  a  common 
format.  Their  development  costs  may  be  reduced  if  common  software  components, 
designed  to  exploit  common  knowledge  formats,  can  be  used  in  different  clinical 
systems. 

Despite  these  potential  benefits,  there  is  little  evidence  of  a  clear  trend  towards 
conceptual  consensus  among  interested  parties,  while  there  are  countervailing  signs  of 
intensifying  competition  between  proponents  of  inferencing  and  authoring  teehnologies 
based  on  rival  KR  models. 


This  impasse  could  be  broken  by  establishing  a  LINUX-like  community  sharing 
development  of  an  open-source  inferencing  component  implementing  the  semantics  of  a 
standard  knowledge  model.  Likely  agencies  with  the  influence  and  authority  to  broker 
and  lead  the  adoption  of  standards  are  public-sector  and  public  interest  guideline 
dissemination  agencies  and  warehouses  (in  the  UK,  such  candidate  agencies  are  the 
National  Institute  for  Clinical  Excellence  and  the  National  Electronic  Library  for  Health). 
Such  a  development  would  generate  expanded  business  opportunities  for  added- value 
products  linking  guideline-driven  inference  to  EHR  platforms,  and  for  creation  of 
advanced  knowledge  authoring  tools. 

Progress  towards  a  common  model  may  to  some  extent  be  partitioned  into  a  set  of  sub¬ 
problems,  such  as: 

a)  An  interface  for  EHR  queries. 

b)  A  grammar  for  expressing  criteria  which  are  capable  of  being  tested  by  means  of 
EHR  queries. 

c)  A  grammar  for  specifying  recommended  healthcare  actions. 

d)  An  algorithm  specification  for  methods  used  to  compute  a  result  or  value,  which  may 
form  a  component  of  a  recommendation  (e.g.  a  drug  dosage,  a  risk  stratification 
score,  a  diagnostic  assessment  conclusion). 

e)  An  object  model  for  expressing  the  compositional  structure  of  a  protocol  and  the 
criteria  governing  the  lifecycle  of  the  use  of  a  protocol  and  its  parts. 

f)  An  interface  for  connecting  a  component  implementing  the  semantics  of  items  (b),(c), 
(d)  and  (e)  and  a  host  clinical  information  environment. 

Consensus  on  any  one  of  the  above  topics  will  begin  to  yield  some  of  the  clinical, 
business  and  public-interest  benefits  identified  above. 

Several  of  these  issues  are  interlinked  with  other  areas  of  current  health  informatics 
standards  work.  Delivery  of  item  (a)  depends  on  standards  for  EHR  architecture.  Delivery 
of  item  (b)  and  depends  on  standards  for  clinical  activity  and  workflow  definition.  Items 
(a),  (c)  and  (e)  all  imply  demands  (some  of  them  novel)  on  the  range  and  expressiveness 
of  clinical  terminology  schemes. 

The  standards  processes  for  EHR  architecture  and  terminologies  have  reached  a  more 
mature  stage  than  those  for  the  topics  discussed  here,  so  that  we  can  share  their  methods 
and  build  on  their  existing  results,  even  if  the  latter  are  not  fully  sufficient  for  our  needs. 
An  important  development  in  recent  CEN  TC251  work  relating  to  the  EHR  has  been  the 
choice  of  (UML-compliant)  model-based,  syntax-independent  formalisms  for 
fundamental  standard  definitions.  This  approach  can  and  should  be  adopted  in  the  present 
discussion  so  that  choices  between  alternative  implementation  syntaxes  (whether 
declarative  or  procedural  in  format)  are  no  longer  dictated  or  demanded  by  the 
standardisation  process.  It  should  be  possible,  as  Avith  the  recent  EHR  models,  to 
automatically  derive  an  XML  DTD  from  a  UML  model-based  standard  definition. 

A  preliminary  solution  for  item  (a)  above  could  probably  be  constructed  on  the  basis  of 
emerging  standards  results  for  EHCR  architecture,  and  by  similar  consensus  processes 
including  both  technical  and  clinical  testing  of  candidate  standards,  with  equivalent 
testing  of  the  integrity  of  knowledge  transfer  across  different  systems.  Any  draft  version 
of  a  standard  comprising  items  (a)  to  (f)  inclusive  above  should  be  accompanied  by  a 
joint  initiative  to  create  an  open-source,  public-domain  software  iriferencing  module 
implementing  their  combined  semantics. 


Knowledge-representation  and  functional  requirements  of  a  shared  clinical  trials 
protocol  format.  Informatics  for  clinical  trials  pose  distinct  logistical,  business  and 
communications  requirements  (as  is  also  the  case  for  the  many  and  various  clinical 
contexts  in  which  non-trials  protocols  are  used),  but  no  distinct  knowledge  representation 
issues. 

Organizational  approaches  to  consensus  development  and  standardization  of 
guideline-sharing  formats.  (Cf  also  the  remarks  above.)  Interest  in  health  informatics 
for  clinical  guidelines  now  far  outscales  the  evidence  base  justifying  any  one  approach  as 
the  way  of  the  future.  Many  interesting  results  exist  but  these  are  all  limited  in  respect  of 
either  (a)  extended  exposure  to  real  clinical  use  (b)  complexity  of  knowledge  content 
supported  (c)  dependencies  on  a  technology  platform  and  host  system  (d)  proof  of  use 
across  multiple  and  varying  environments.  Over  and  above  software  and  tool  creation,  the 
task  of  knowledge  authoring  is  a  potentially  crippling  overhead  and  investment  in 
increasingly  sophisticated  tools  has  not  yet  been  shown  to  reduce  the  difficulty  of  the 
authoring  process  and  the  concomitant  need  for  scarce  and  unavailable  levels  of 
authoring  skill.  Moreover,  there  is  as  yet  no  proof  that  the  healthcare  community  is 
capable  of  mastering  the  combined  organizational  and  technological  challenges  of  safely 
maintaining  and  managing  over  time  large  digital  corpuses  of  complex  shared 
knowledge. 

For  all  these  reasons,  approaches  to  consensus  development  and  standardisation  should 
proceed  cautiously  and  with  deliberation. 


Clinical  Guidelines  Representation: 

Incrementally  Beyond  Arden  Syntax 


Robert  A.  lenders,  MD,  MS 

Department  of  Medical  Informatics,  Columbia  University 
Co-Chairman,  HL7  Clinical  Decision  Support  and  Arden  Syntax  Technical  Committee 


Context 

Although  much  effort  has  been  expended  to  create  clinical  guidelines,  use  of  and  compliance  with 
guidelines  is  felt  to  be  less  than  ideal.  Many  people  believe  that  presenting  tailored,  guideline- 
based  decision  support  at  the  point  of  care  using  computers  will  improve  compliance.  However, 
some  argue  that  current  standards  for  guideline  representation  may  hinder  such  implementations 
and  impair  knowledge  sharing.  Arden  Syntax  is  an  ANSI  standard  for  such  knowledge  sharing, 
but  some  argue  that  it  does  not  adequately  allow  guideline  representation.  Other  efforts  (GEODE, 
EON,  GLIF,  etc)  have  been  made  to  provide  a  knowledge  representation  formalism  for  clinical 
guidelines. 

Approach 

The  HL7  Clinical  Decision  Support  and  Arden  Syntax  Technical  Committee  discussed  the  issue  during  its 
January,  2000  meeting.  The  technical  committee  concurred  with  the  following  conclusions. 

Summary  Points 

1.  Arden  Syntax  is  the  only  widely  accepted  standard  for  clinical  knowledge  representation.  A 
number  of  system  vendors  (e.g.,  listed  alphabetically:  Eclipsys,  HBOC,  SMS)  and  some  knowledge 
vendors  (e.g.,  Micromedex)  incorporate  Arden  Syntax  in  their  products.  The  installation  base  of  Arden  is 
growing.  Arden  is  sponsored  by  an  SDO  (HL7)  that  is  thriving  and  which  will  persist.  Therefore,  any 
future  standard  should  incorporate  or  use  Arden  as  much  as  possible.  Further,  Arden  Syntax  offers  a 
number  of  features  (e.g.,  operators  for  temporal  reasoning)  that  make  it  appropriate  for  clinical  decision 
support. 

2.  Clinical  guidelines,  even  complex  ones,  can  be  represented  in  Arden  now  without  further  altering 
the  Syntax.  However,  some  other  formalisms  (e.g.  GLIF)  explicitly  declare  inclusion/exclusion  criteria  as 
well  as  the  clinical  states  and  their  transitions  for  an  entire  guideline  (instead  of  a  loosely  aggregated 
collection  of  MLMs).  These  constructs  could  be  added  to  Arden  to  improve  guideline  representation. 

3.  Any  guideline  representation  that  promotes  knowledge  sharing  will  have  to  address  the  problem 
of  event  deflnitions,  clinical  vocabularies  and  database  schemata  that  differ  from  site  to  site.  Arden 
Syntax  currently  does  not  offer  a  standard  way  to  accomplish  these  things,  but  the  Reference  Information 
Model  of  HL7  (RIM)  is  a  robust  and  growing  candidate  for  a  standard  data  model.  Accordingly, 
involvement  of  HL7  will  be  important  to  this  effort. 

4.  To  improve  shareability,  a  guideline  representation  will  have  to  provide  a  mechanism  for 
communicating  tailored  messages  to  personnel.  The  Arden  Syntax  TC  is  actively  working  on  a  standard 
syntax  for  this  process. 


5.  Some  people  have  criticized  Arden  Syntax  as  suffering  from  the  commonly-cited  drawback  of  rule- 
based  expert  systems:  the  inability  to  regulate  the  order  of  rule/MLM  execution.  However,  MLMs  can  be 
explicitly  chained  using  CALL  statements,  thus  helping  to  overcome  this  issue. 


6.  Potential  roles  for  Arden  Syntax  (neither  mutually  exclusive  nor  exhaustive): 

(a)  It  continues  as  it  is  without  further  alteration  to  accommodate  guidelines.  A  standard  guideline 
representation  (e.g.,  GLIF)  would  be  translated  to  Arden  MLMs  at  eaeh  site  in  an  automated  way. 
Eaeh  MLM  might  represent  a  single  “step”  in  a  GLIF-encoded  guideline.  In  this  formulation,  a 
GLIF  or  other  type  of  “wrapper”  would  organize  a  collection  of  MLMs  into  a  complete  guideline. 
Allowing  such  modular  guideline  components  would  promote  knowledge  reuse  as  well. 

(b)  It  incorporates  guideline  constructs  (e.g.,  explicit  definition  of  steps  and  transitions)  and  thus 
serves  as  a  eomplete  alternative  to  GLIF  and  other  representations. 

General  Conclusions 

Because  of  the  growing  use  of  Arden  Syntax  and  investment  in  information  systems  that  incorporate  it, 
future  efforts  to  represent  guidelines  should  leverage  this  experience  and  use  the  Syntax  as  much  as 
possible.  Arden  Syntax  will  benefit  from  incremental  change  to  incorporate  guideline  constructs.  In  turn, 

Arden  Syntax  could  serve  as  a  major  component  of  a  future  guideline  representation  formalism. 


Topic:  Knowledge  Representation  of  Situation-Specific  Clinical  Practice 

Guidelines 

Submitted  by:  Ronilda  C.  Lacson,  MD 

Clinical  practice  guidelines  (CPGs)  are  systematically  developed  statements 
developed  to  assist  practitioner  and  patient  decisions  about  appropriate  health  care  for 
specific  clinical  circumstances.'  The  main  goal  is  to  reduce  variability  in  clinical  practice 
while  following  an  algorithm  that  is  evidence-based.  Adherence  to  these  guidelines  is 
expected  to  reduce  costs  of  medical  care  while  improving  patient  outcomes.^’^ 

The  evidence-based  medicine  work-group  published  an  approach  to  guideline 
evaluation  Questions  that  they  suggested  users  ask  include  “What  are  the  risks  and 
benefits?”,  “How  do  these  compare  in  different  people  and  with  different  screening 
strategies?”,  and  “What  is  the  impact  of  people’s  values  and  preferences?”. 

The  three  main  goal  of  this  statement  is  to  recommend  that  a  novel  knowledge 
representation  tool  for  CPGs  should  explicitly  include  the  following: 

1 .  Patient  preferences  and  utilities, 

2.  Mechanism  to  represent  and  compare  various  competing  options  with  regards  to 
patient  outcomes,  costs,  and  estimated  gain  from  adherence,  and 

3 .  Strength  of  evidence  for  each  specific  recommendation. 

Patient  Preferences  and  Utilities 

Medical  decision-making  often  incorporates  knowledge  of  the  medical  domain, 
results  of  published  research,  physicians’  experiences  and  heuristics,  patient  preferences 
and  quality  of  life  issues.  Decision  analysis  is  uniquely  important  because  it  represents 
Quality  Adjusted  Life  Years  (QALYs)  and  utilities  for  treatment  selection.  This  enables 
modeling  of  situation-specific  variables  in  a  decision-making  process.  Therefore,  it  is 
possible  to  apply  decision-analytic  tools  to  guideline  representation. 

Recent  models  that  have  influenced  the  practice  of  medicine  include 
recommendations  for  estrogen  replacement  therapy  in  women,  treatment  options  for 
hepatitis  C,  and  use  of  screening  mammography  in  elderly  women.^’^  There  has  been  an 
effort  in  the  past  to  use  decision  tables  to  convert  probabilistic  data  from  decision  trees 
into  clinical  algorithms.*  Rule  sets  are  identified  that  become  specific  recommendations 
in  a  CPG.  In  the  absence  of  decision  tables,  text  recommendations  or  computerized 
decision  support  tools  are  available  based  on  results  of  these  studies. 

Comparison  of  Various  Care  Options 

It  is  important  to  address  that  CPGs  may  include  recommendations  that  are 
appropriate  for  specific  situation  and  patient  populations.  It  is  therefore  necessary  to 
model  the  competing  options  as  well  as  the  risks  and  benefits  of  each  one.  Cost- 
effectiveness  studies  address  these  issues.^’’  An  analysis  of  the  incremental  cost- 


effectiveness,  gain  in  quality  adjusted  life  years,  and  other  patient  outcomes  are  important 
factors  to  know  as  well.  However,  it  should  be  easy  to  represent  choices  in  a  decision¬ 
making  process  and  make  explicit  the  reason  for  a  specific  recommendation,  especially 
with  the  powerful  tools  currently  available. 

Strength  of  Evidence 

Evidence-based  recommendations  are  better  followed  in  practice  than 
recommendations  not  based  on  scientific  evidence.®  Physicians  and  other  guideline  users 
are  interested  in  the  source  of  evidence  and  the  decision-making  process  that  led  to  a 
recommendation.  A  randomized  clinical  trial  (RCT),  for  example,  that  compares 
alternative  treatment  strategies  is  usually  rated  highly  as  an  evidence  source.  However, 
there  are  not  enough  RCTs  available  for  medical  decision-making  and  the  information 
usually  apply  to  specific  patients  in  specific  situations.  The  strength  of  evidence  may 
depend  on  other  published  clinical  trials  (and  the  strength  of  their  design)  or  on  other 
methods  to  combine  the  results  of  several  studies.  The  latter  method  may  involve  meta¬ 
analysis,  decision  analysis,  or  a  consensus  process  based  on  actual  evidence. 

Summary 

Creating  a  model  for  explicit  representation  of  the  decision-making  process  in 
CPGs  is  the  major  goal  of  this  position  paper.  This  has  not  been  specified  in  current  CPG 
models.  The  actual  decisions  or  recommendations  that  a  CPG  provides  usually  include 
this  decision-making  process,  albeit  implicitly.  Thus,  it  will  not  be  an  added  burden  to 
the  overall  guideline  development  process. 

In  addition,  the  explicit  model  will  enable  supporting  information  about  CPGs  to 
be  stated  specifically.  The  users  will  have  a  better  understanding  of  reasons  why  speeific 
options  are  better  than  others.  It  gives  the  users  more  flexibility  in  following  local 
practices  when  existing  recommendations  support  these.  Furthermore,  the  flexibility  also 
encourages  evidence-based  practice  because  users  are  forced  to  justify  reasons  explicitly 
for  specific  choices. 

Another  asset  of  this  model  is  the  ease  in  representing  changes  for  updating 
CPGs.  When  there  are  changes  in  disease  prevalence  or  when  new  technology  becomes 
available,  it  would  be  easier  to  update  information  in  the  model. 

Cost-effectiveness  analysis  can  also  be  useful  in  the  current  practice  of  medicine. 
This  is  important  when  several  options  are  equally  preferable,  but  differ  signifieantly  in 
cost. 


The  creation  of  a  situation-specific  and  patient-specific  model  is  needed  for 
medical  decision-making.  As  previously  noted,  the  evidence-based  medicine  working 
group  supports  this  approach  in  evaluating  CPGs.'*  It  is  certainly  closer  to  how  decision¬ 
making  is  performed  in  real  clinical  situations.  Local  adaptation  of  a  “centralized 
guideline”  may  be  feasible  in  this  context. 
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Clinical  practice  guidelines  have  been 
developed  and  used  for  multiple  aspects 
of  patient  care,  including  referral, 
utilization,  disease  management  and 
preventive  care  (1).  In  many  of  these 
practice  recommendations,  it  is  not 
uncommon  to  find  situations  where  an 
intervention,  a  referral,  or  a  preventive 
care  approach  is  contraindicated  or 
strongly  not  recommended.  The  main 
goal  of  this  statement  is  to  emphasize  the 
need  for  a  guideline-authoring  tool  that 
can  adequately  represent  and  implement 
“negative”  recommendations.  Specific 
areas  of  concern  are  noted  below. 

Health  Care  Resource  Utilization 
In  the  era  of  cost  containment,  it  is 
necessary  to  make  decisions  about 
adequate  and  appropriate  use  of  medical 
technology.  Cost-effectiveness  studies 
are  entering  the  mainstream  of  clinical 
and  scientific  research.  Several  studies 
support  the  utilization  of  diagnostic 
exams  for  specific  situations  only.  For 
example,  getting  an  electrocardiogram 
(ECG)  for  patients  less  than  30  years  of 
age  with  no  risk  factors  during  routine 
preoperative  evaluation  is  found  to  be 
cost-ineffective  (2).  Although  guidelines 
specifically  do  not  recommend  an  ECG 
to  be  done  in  this  particular  situation, 
inability  to  represent  this  negative 
recommendation  may  suggest  that  this 
was  not  a  part  of  the  guideline  or  that  the 
authors  failed  to  address  this  issue. 
Either  way,  the  point  is  missed  and  a 
user  may  decide  to  get  the  diagnostic 
exams  anyway.  Another  important 


application  of  negative 

recommendations  concerns  referral  and 
consultation  with  sub-specialists.  To 
illustrate,  a  gastroenterology  referral  (for 
endoscopy)  for  patients  with  dyspepsia 
may  not  always  be  indicated  (3). 
Appropriate  recommendations  are  often 
specified  in  guidelines  for  this  purpose. 
Furthermore,  omission  of  negative 
recommendations  may  harm  patients  in 
some  cases.  The  American  Heart 
Association  specifically  enumerates 
situations  when  endocarditis  prophylaxis 
is  not  recommended  in  their  scientific 
statement  (4).  Inappropriate  use  of 
antibiotics  for  endocarditis  prophylaxis 
may  affect  antibiotic  susceptibility 
patterns  within  specific  institutions  or 
potentially  cause  allergic  reactions. 

Inappropriate  Combinations 

In  an  alerting  system,  the  presence  of  a 
potentially  harmful  drug  combination 
may  trigger  a  warning  to  the  user,  often 
evoked  through  a  medical  logic  module 
(5).  Current  guideline  representation 
formats  do  not  allow  inclusion  of 
potentially  harmful  drug  interaction. 
Furthermore,  treatment  with  an 
inappropriate  drug  (or  combination  of 
drugs)  such  as  single-drug  regimen  for 
the  treatment  of  H.  pylori  can  not  be 
adequately  expressed  (3). 

Contraindications 

A  contraindication  is  defined  as  a 
symptom  or  condition  that  makes  a 
particular  treatment  or  procedure 
inadvisable  (6).  For  example,  in  the 


practice  parameter  for  the  management 
of  acute  gastroenteritis  in  children 
(published  by  the  American  Academy  of 
Pediatrics),  several  drugs  were 
specifically  listed  as  either  “not 
recommended”  or  “contraindicated”  for 
the  treatment  of  acute  diarrhea  (7).  In 
fact,  as  a  general  rule,  they  specifically 
stated  that  “pharmacological  agents 
should  not  be  used  to  treat  acute 
diarrhea.”  This  needs  to  be  adequately 
addressed  and  represented. 

Timed  Interventions 

Finally,  it  may  also  be  prudent  to 
indicate  time  intervals  when  specific 
recommendations  are  applicable.  This  is 
another  aspect  of  evidence-based  care 
often  specified  in  cost  effectiveness 
studies.  Appropriate  intervals  for 
cervical  cancer  screening  using  pap 
smears  or  the  frequency  of 
mammography  for  breast  cancer 
screening  will  need  to  be  represented. 
Inappropriate  under-utilization  as  well  as 
over-utilization  of  these  and  other 
preventive  examinations  at  intervals  that 
are  more  or  less  frequent  than  necessary 
should  prompt  a  reminder  to  the  user 
that  the  action  is  not  supported  by  the 
current  guideline.  The  same  is  true  for 
ordering  laboratory  tests  at  intervals 
more  frequent  than  necessary.  This  is 
exemplified  by  the  disutility  of  more 
frequent  serum  monitoring  for 
hyperkalemia  in  hemodialysis  patients 
on  erythropoietin  (8). 

Limitation 

A  potential  danger  with  this  approach  is 
over-representation  of  clinical  practice 
guideline  recommendations.  When  a 
guideline  specifies  conditions  whereby  a 
referral  is  appropriate,  does  that  imply 
that  every  other  condition  is 
inappropriate?  Furthermore,  when  data 


is  lacking  to  specify  recommended 
intervals  for  screening  or  interventions, 
are  expert  opinions  valid?  Conversely, 
the  possibility  of  being  over-cautious 
exists.  How  does  one  determine  when  a 
positive  recommendation  implies  that 
any  other  approach  would  not  be 
appropriate?  However,  I  do  not  think 
that  these  are  issues  to  be  addressed  by  a 
guideline  representation  tool.  These  are 
issues  for  the  guideline  authors.  What  is 
important  for  us  is  the  ability  to 
accommodate  when  guidelines  contain 
negative  recommendations  that  are 
specifically  stated.  The  authoring  tool 
we  develop  should  be  able  to  represent 
this. 

In  summary,  I  have  expounded  on  the 
need  for  representation  of  “negative” 
guideline  recommendations,  an 
important  issue  not  to  be  overlooked. 
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Clinical  practice  guidelines  are  a  mechanism  for  translating  evidence  from 
scientific  research  into  clinical  practice  with  an  intended  goal  of  improving  health  care 
outcomes.  The  process  of  using  guidelines  to  affect  clinical  practice  is  complex  and  can 
be  more  clearly  conceptualized  if  divided  into  several  component  steps.  Delineation  of 
such  discrete  steps  creates  a  fiamework  that  facilitates  the  sharing  of  guideline  content 
and  the  methodologies  that  promote  guideline  use.  In  addition,  since  new  scientific 
evidence  is  continually  becoming  available,  such  a  framework  provides  a  mechanism  by 
which  guidelines  can  be  systematically  updated. 

A  proposed  model  representing  the  steps  involved  in  using  guidelines  in  clinical 
practice  is  shown  in  Figure  1.  In  this  model,  guidelines  are  conceptualized  as  existing  in 
a  dynamic  multi-step  “life  cycle.”  This  model  is  cyclical  because  guidelines  are  viewed 
as  undergoing  periodic  revision  to  incorporate  new  knowledge.  Six  discrete  steps  are 
identified  which  include:  creation,  dissemination,  implementation,  utilization,  evaluation, 
and  modification.  This  life-cycle  model  illustrates  how  scientific  research  evidence  is 
incorporated  into  the  development  and  on-going  modification  of  guidelines.  It  also 
portrays  the  ultimate  goal  of  guideline  use,  which  is  to  improve  health  care  outcomes. 

The  life  cycle  model  begins  with  the  creation  of  the  guideline.  Creation  denotes 
the  process  of  identifying  and  distilling  medical  knowledge  pertaining  to  a  specific 
clinical  scenario  into  recommendations  for  its  management.  The  broken  line  in  the  model 
between  scientific  evidence  and  creation  highlights  the  importance  of  sound  scientific 
evidence  as  the  basis  for  clinical  practice  guidelines.  In  the  second  step,  after  a  guideline 
is  created,  it  is  disseminated  to  its  intended  users.  Dissemination  denotes  the  processes 
by  which  a  guideline  is  promulgated  throughout  the  intended  user  community.  Once  in 
the  hands  of  the  intended  users,  the  guideline  must  be  implemented  into  the  clinical 
setting,  which  is  the  third  step  of  the  life  cycle.  Implementation  is  the  process  of  making 
information  from  the  guideline  available  in  a  clinical  context  where  this  information  can 
influence  the  care  process.  Implementation  can  range  from  posting  a  printed  copy  of  a 
guideline  on  an  examination  room  wall  to  elaborate  computer-based  solutions.  After 
guideline  information  is  available  in  the  clinical  setting,  it  must  be  utilized.  The  process 
of  using  guideline  information  to  direct  health  care  is  step  four  in  this  life  cycle  model 
and  is  termed  utilization. 

Although  utilization  leads  to  the  desired  endpoint  of  influencing  health  care 
outcomes,  the  life  cycle  model  does  not  end  here.  It  is  critical  for  the  guideline  to 
progress  through  two  additional  steps.  Step  5  in  the  life  cycle  is  evaluation.  Evaluation 


denotes  two  processes:  first,  the  evaluation  of  the  outcomes  that  result  from  following  the 
guideline  recommendations,  i.e.,  do  patients  really  do  better  as  a  result  of  using  this 
guideline;  second,  the  analysis  of  each  step  of  the  guideline  life  cycle  to  determine 
whether  or  not  the  methods  selected  to  accomplish  the  step  were  effective.  Finally,  step  6 
of  the  life  cycle  allows  for  the  guideline  to  be  modified.  Modification  refers  to  the 
process  of  updating  or  revising  a  guideline  based  on  new  scientific  evidence  or  on  what 
has  been  learned  through  the  guideline  evaluation  process.  Guideline  modification  is 
critical  for  the  long-term  viability  of  the  guideline  in  clinical  practice.  After  modification, 
it  follows  that  the  guideline  is  re-disseminated,  re-implemented  and  so  on  to  have  a 
continued  effect  on  health  care  outcomes. 

In  summary,  this  model  provides  a  representation  of  guidelines  as  dynamic 
components  of  the  care  process  that  systematically  change  to  incorporate  new  knowledge. 
Introduction  of  this  dynamic  aspect  of  guidelines  addresses  some  issues  of  “cookbook 
medicine”  ascribed  to  guidelines  and  the  inflexibility  often  associated  with  guideline- 
based  practice.  This  dynamism  also  leads  to  a  new  set  of  challenges  and  opportunities  in 
the  development  and  sharing  of  guidelines  for  the  improvement  of  health  care  outcomes. 


Figure  1.  The  Clinical  Practice  Guideline  Life  Cycle 
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Overview 


Medicalogic  is  committed  to  providing  healthcare  professionals  with  tools  to  provide  them  with  a  broad 
scope  of  decision  support  functionality.  Currently  we  provide  them  with  the  ability  to: 

•  Implement  practice  guidelines. 

•  Design  and  generate  longitudinal  administrative  and  clinical  reports. 

•  Customize  and  implement  entry  forms 

•  Create  protocols  that  can  be  triggered  by  time  as  well  as  various  types  of  events,  e.g.  immunization 
reminders. 

•  Create  ad  hoc  querying  ability. 

•  Support  point-of-care  (preemptive)  decision  support  functionality,  e.g.  drug  allergy  and  drug-drug 
interactions. 

Logician  provides  a  variety  of  tools  available  to  those  interested  in  impacting  the  practice  of  medicine. 
Taken  together,  these  tools  can  provide  a  powerful  armamentarium  for  effecting  change. 


Current  Components 


Encounter  Forms 

Encounter  forms  are  the  primary  mechanism  by  which  users  document  a  clinical  encounter.  Users  who 
create  these  forms  decide  what  information  captured  with  the  form  is  stored  as  discrete  data  and  what 
simply  contributes  to  the  note.  In  addition,  form  creators  can  include  guideline  related  static  information 
on  disease  specific  forms  to  reminder  providers.  The  results  of  protocols  can  be  easily  displayed  on  forms 
to  impact  point-of-care  decision-making. 

Protocols 


Protocols  can  be  implemented  within  a  chart  to  guide  and  remind  a  practitioner  of  orders  that  are  due  for  a 
patient.  These  are  typically  driven  by  dates  as  well  as  various  other  clinical  events. 

Inquires 

Inquiries  allow  providers  to  quickly  create  powerful  ad  hoc  queries  within  or  across  charts  that  would 
otherwise  require  many  hours  of  manual  labor. 

Reports 

Design  and  implementation  of  reports  allow  quick  and  useful  aggregation  of  data  across  charts  for  clinical 
as  well  as  administrative  analysis.  Stored  reports  can  be  generated  on  a  regular  basis  to  assess  quality  of 
care,  utilization,  productivity  benefits  and  reimbursement. 


Formularies 

Medicalogic  allows  users  to  implement  formularies  to  direct  prescribers  to  more  cost  effective  medications. 
We  provide  our  users  with  a  Formulary  Editor  as  well  as  provide  a  published  syntax  which  allow  our  users 
to  use  third  party  formulary  data  (InfoScan)  and  import  directly  into  Logician. 

Open  architecture 

For  the  past  three  years,  MedicaLogic  has  taken  the  position  that  clinical  content,  including  protocols, 
electronic  forms,  reports,  formularies,  etc.,  should  be  sharable  across  Logician  users.  To  this  end. 


MedicaLogic  set  up  the  KnowledgeBank  (see 

httD://knowledqe. medicaloqic.com/carbo. dll?icatcommand=knowldge&catalogname=KB  Final)  as 


a  forum  for  our  users  to  share  clinical  content.  This  content  is  created  either  within  Logician  or  using 
separate  tools  available  to  our  users. 

Terminology 

Medicalogic  manages  a  set  of  data  elements  (observations)  used  by  our  users  to  capture  discrete  clinical 
data  outside  of  problems,  medications,  allergies,  and  orders.  Users  make  requests  for  new  terms  and  these 
requests  are  processed  by  Medicalogic.  In  addition,  Medicalogic  is  developing  a  new  unified  terminology 
strategy  to  support  our  internet  products  (Logician  Internet  and  98point6 — consumer  health  record  site)  as 
well  as  our  client-server  product.  Logician  Enterprise.  This  new  terminology  model  will  support  all  of  our 
new  reporting  and  decision  support  offerings. 


We  provide  our  user  with  documentation  and  the  relational  database  schema  thereby  enabling  sites  to  use 
third  party  reporting  and  analysis  tools  to  answer  questions. 

Future  Directions 

Web-based  reporting 

We  have  a  current  initiative  geared  towards  allowing  our  Logician  Internet  users  to  quickly  view  aggregate 
data  for  their  practices.  In  addition,  the  first  phase/implementation  provides  healthcare  professionals  with 
tools  that  allow  them  to  implement  practice  guidelines  within  a  single  practice  or  across  an  enterprise.  The 
professional  has  access  to  standard  guidelines  fi'om  well-established  professional  organizations  and  the 
ability  to  view  and  customize  them  to  suite  his/her  practice. 

General  purpose  rules  engine 

The  current  protocol  engine  has  limitations.  We  believe  that  a  general  purpose  rules  engine  is  required  that 
would  allow  the  identification  of  triggers  (patient  checking-in,  abnormal  lab  test  returning,  ete.)  and  actions 
(sending  an  email,  beeping  a  provider,  etc.)  as  well  as  a  more  flexible  rules  syntax.  We  have  been  following 
the  Arden  Syntax  effort  and  we  are  open  to  exploring  the  utility  of  Arden  in  this  context. 

Extensions  to  Web-based  reporting 

Future  enhancements  to  the  web-based  reporting  initiative  will  include  real  time  test  performance, 
assessment  for  a  given  individual  decision  based  upon  population  data,  local,  regional,  and  national 
comparisons;  best  practice  assessment  and  population  monitoring. 
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Section  1 13  of  the  Food  and  Drug  Administration  (FDA)  Modernization  Act  of  1997 
requires  the  establishment  of  a  public  resource  of  information  on  clinical  trials,  whether 
federally  or  privately  funded,  for  experimental  treatments  for  serious  or  life-threatening 
diseases  and  conditions  [1].  The  responsibility  for  creating  the  database  was  given  to  the 
National  Institutes  of  Health  (NIH)  and  in  September  of  1998,  Harold  Varmus,  the 
former  director  of  NIH,  asked  the  National  Library  of  Medicine  (NLM)  to  take  the  lead  in 
the  project.  Given  that  the  scope  of  the  legislation  is  extremely  broad,  including  not  only 
NIH,  but  also  other  federally  sponsored  trials,  as  well  as  privately  sponsored  trials,  we 
decided  to  approach  the  task  in  phases.  The  initial  phase  would  involve  building  the 
system  to  include  primarily  NIH  sponsored  trials  and  subsequent  phases  would 
incorporate  trials  from  other  federal  agencies  and  the  private  sector.  The  first  phase  is 
coming  to  an  end  with  the  public  release  of  the  first  version  of  the  system  in  early 
February  2000  [2]. 

The  system,  called  ClinicalTrials.gov,  currently  contains  more  than  4,000  records  and  has 
been  designed  to  provide  patients,  families,  and  other  members  of  the  public  easy  Web- 
based  access  to  information  about  clinical  research  studies.  Each  record  in  the  database 
includes  a  summary  of  the  purpose  of  the  trial,  together  with  its  recruiting  status,  the 
criteria  for  patient  participation,  the  location  of  the  trial,  and  specific  contact  information. 
Other  information  in  the  database  that  may  help  a  patient  decide  whether  to  enroll  in  a 
particular  trial  includes  the  research  study  design,  the  phase  of  the  trial,  the  disease  or 
condition,  and  the  particular  drug  or  therapy  under  study.  An  important  feature  of  the 
database  is  that  it  provides  access  to  other  online  health  resources,  such  as  MEDLINE 
and  MEDLINEplus,  that  help  place  clinical  trials  in  the  context  of  patients'  overall 
medical  care. 

When  we  began  this  project  it  seemed  clear  that,  since  data  would  be  coming  to  us  from 
many  different  sources,  the  first  step  would  be  to  reach  agreement  on  a  common  set  of 
data  elements.  In  the  fall  of  1998  we  met  with  representatives  from  each  of  the  twenty- 
one  institutes  who  conduct  or  sponsor  clinical  trials  to  discuss  a  proposed  set  of  data 
elements.  By  the  end  of  the  year  we  had  agreed  on  about  a  dozen  required  elements  and 
an  equal  number  of  optional  elements.  Our  deliberations  were  informed  by  earlier  work 
that  had  been  done  at  NIH,  including  work  on  existing  disease-specific  trials  registries, 
such  as  the  AIDSTRIALS  database  which  was  created  some  ten  years  ago  as  a 
collaboration  between  NLM,  the  National  Institute  of  Allergy  and  Infectious  Diseases, 
and  the  FDA.  Further,  we  were  aware  of  other  efforts  in  bodi  the  informatics  and  clinical 
communities  on  the  development  of  clinical  trials  systems,  e.g.,  [3-7].  Our  current  set  of 


required  data  elements  includes  a  unique  study  identifier,  a  title,  summary,  recruitment 
status,  eligibility  criteria,  study  type  and  design,  and  location  and  contact  information. 
Optional  data  elements  include  references  for  ongoing  or  completed  trials,  results, 
keywords,  and  supplementary  information,  such  as  related  URL’s.  (For  example,  if  the 
trial  is  about  diabetes,  a  related  site  might  be  the  National  Diabetes  Education  Program.) 

Once  we  had  agreed  on  the  basic  set  of  data  elements,  we  needed  to  work  with  each  NIH 
group  individually  in  the  implementation  of  our  plans.  Our  model  is  that  of  a  centralized 
database  to  which  XML-tagged  data  are  regularly  submitted  by  multiple  data  providers. 
The  data  are  FTP’ed  to  our  site,  and  then  we  validate  and  enhance  the  data  for  inclusion 
in  the  publicly  available  system  which  is  updated  nightly.  Since  our  initial  set  of  data 
providers  (the  21  institutes)  differed  in  their  technical  infrastructure,  technical  expertise, 
and  in  some  cases  only  had  their  trial  data  in  paper  form,  we  needed  to  devise  multiple 
ways  for  them  to  be  able  to  contribute  to  the  project.  Some  had  existing  databases  that 
contained  clinical  trials  data  and  in  that  case  our  task  was  to  work  with  them  to  ensure 
that  they  could  generate  a  report  that  was  consistent  with  our  DTD.  Even  if  a  database 
existed,  however,  sometimes  the  semantics  of  some  of  the  data  elements  were  not 
consonant  with  ours,  and  so  we  assisted  in  the  evaluation  and  possible  restructuring  of 
portions  of  their  databases.  For  those  who  did  not  have  a  database,  we  created  a  Web- 
based  data  entry  system  that  would  allow  them  to  enter  their  data  de  novo.  Those  data 
come  to  an  ancillary  database  here  at  NLM,  where  it  is  stored  and  managed,  but  the 
ultimate  content  control  still  resides  with  the  responsible  individual  at  the  institute.  Some 
other  NIH  groups  took  this  project  as  an  opportimity  to  develop  the  needed  technical 
infrastructure  and  are  now  able  to  deliver  the  data  to  us  from  their  databases. 

Perhaps  the  most  important  lesson  we  have  learned  is  that  in  order  to  effect  a  project  of 
this  scope,  and  one  that  involves  such  large  numbers  of  individuals,  is  that  even  if  there  is 
general  agreement  on  standards,  the  devil  is  truly  in  the  details.  The  importance  of 
educating  individuals  about  the  value  and  necessity  of  standards  cannot  be  over¬ 
emphasized,  and  one  must  be  willing  to  invest  the  necessary  time  and  effort  to  do  so  and 
to  give  them  assistance  in  implementing  those  standards  when  they  need  it. 
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A  challenge  for  the  future  will  be  to  develop  strategies  and  tools  to  facilitate  maintaining  complex  computer-based 
clinical  guidelines  as  the  underlying  knowledge  evolves  over  time.  A  standardized  guideline  representation  could 
facilitate  the  development  of  tools  and  strategies  that  could  be  shared,  at  least  in  part,  by  many  different  guidelines. 

Childhood  Immunization  We  are  exploring  these  issues  in  the  context  of  IMM/Serve,  a  computer-based  guideline 
for  childhood  immunization.  IMM/Serve  takes  as  input  a  child’s  immunization  history  and  produces 
recommendations  as  to  which  vaccinations  are  due  and  which  should  be  scheduled  next  (and  when).  IMM/Serve’s 
knowledge  base  (KB)  is  expressed  using  if-then  rules,  tabular  parameters,  and  procedural  logic.  IMM/Serve  is  being 
used  operationally  in  roughly  10  geographically-dispersed  beta  sites  within  the  US  Indian  Health  Service  (IHS), 
with  planned  use  at  roughly  200  IHS  sites  nationwide  in  the  near  future. 

Maintaining  a  Record  of  KB  Changes  We  are  maintaining  a  record  of  all  changes  made  to  the  IMM/Serve’s  KB. 
These  changes  may  be  made  for  several  reasons,  including  1)  changes  in  the  recommendations  of  national  panels,  2) 
changes  in  the  local  interpretation  of  the  national  guidelines,  3)  local  customization,  and  4)  identification  of  errors 
and  inadequacies  in  the  logic.  Over  the  past  year,  there  have  been  significant  KB  changes  for  each  of  these  reasons. 

Strategies  and  Tools  for  Knowledge  Maintenance  We  are  developing  a  suite  of  strategies  for 
validating  IMM/Serve’s  evolving  KB,  including  tools  for  automatic  generation  of  test  cases 
using  domain-specific  approaches  (tailored  to  immunization)  and  using  more  generic  approaches 
(which  could  be  used  for  other  clinical  guidelines).  We  are  exploring  the  potential  “fit”  between 
the  various  strategies  and  tools  and  the  actual  KB  changes  made  over  the  past  year. 

A  Planned  Web  Site  We  plan  to  maintain  a  Web  site  which  will  contain  1)  “snapshots”  of  successive  stable 
versions  of  IMM/Serve’s  M,  2)  the  corresponding  operational  versions  of  IMM/Serve  which  can  be  used  for 
research  purposes  over  the  Web,  and  3)  a  description  of  the  changes  made  to  each  version  as  described  above. 

Potential  Benefits  of  a  Standardized  Guideline  Representation  A  standardized  guideline  representation  could 
provide  several  benefits.  It  would  facilitate  creation  of  computer-based  tools  for  knowledge  maintenance  that  could 
be  shared  by  different  guidelines.  It  could  allow  the  KB  “snapshots,”  and  the  changes  made  to  each,  to  be  described 
in  a  format  that  would  be  more  accessible  to  Medical  Informatics  researchers  interested  in  exploring  the  nature  and 
implications  of  such  changes. 
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Two  topics  are  being  investigated  at  present: 


Consensus  Development  of  Guidelines 

There  are  many  areas  of  medicine  where  evidence  based  guidelines  are  difficult  to  develop. 
Difficulties  in  developing  guidelines  can  arise  due  to  the  harmful  nature  of  controlled  studies 
and/or  the  low  incidence  of  a  certain  problem  requiring  impossibly  large  study  populations.  In 
these  cases,  development  of  consensus-based  guidelines  is  perhaps  the  next  best  approach.  The 
web  presents  a  method  for  the  rapid  development  of  consensus  based  guidelines  with  an 
international  scope  and  presumably  void  of  the  biases  present  in  smaller  efforts  (e.g.  from 
professional  societies). 

We  are  fimded  by  NIH  to  develop  a  method  for  developing  guidelines  using  an 
international  group  of  experts  and  an  interactive  guidelines  web  site.  The  medical  topic 
deals  with  various  forms  of  neurological  monitoring  in  surgery  and  critical  care.  We  are 
investigating  the  role  of  expert  groups  (e.g.  entry  criteria),  moderators  for  a  specific 
guideline,  and  user  interfaces.  The  tools  being  developed  are  general  purpose  and  can  be 
used  in  the  development  of  other  guidelines.  One  challenge  is  to  develop  the  guidelines  in 
a  method  consistent  with  the  Guideline  Interchange  Format. 

The  Bedside  Markup  Language  (BML) 

Most  clinical  personnel  agree  that  it  is  next  to  impossible  to  obtain  at  the  bedside  all  the 
information  that  may  be  relevant  to  the  management  of  a  particular  patient.  This  information  goes 
beyond  guidelines  and  includes  general  medical  information,  drug  details,  device  instructions, 
policies,  etc.  The  web  site  of  the  medical  publisher  Mosby  claims,  "According  to  a  recent  study, 
health  care  persoimel  have  a  significant  question  that  goes  imanswered  for  every  two  to  four 
patients.  A  bounty  of  resources  exists  for  health  care  professionals  to  answer  urgent  medical 
questions.  Usually  the  information  is  not  available  where  it's  needed  the  most  —  at  the  point  of 
care."  One  reason  for  this  problem  is  the  variety  of  formats  of  bedside  information  that  may  be  in 
textbooks,  product  manuals,  policy  manuals,  videotapes,  CD-ROMs,  web  pages,  etc.  These 
various  formats  almost  prevent  it  fi'om  being  used  quickly  at  the  bedside  where  it  is  needed. 

In  these  examples  of  bedside  information,  the  "content",  is  coupled  to  the  respective 
"display  and  distribution"  mechanism.  For  example,  Mosby's  GenRx  is  a  drug  database 
(the  "content")  which  you  buy  as  a  CD-ROM  ("delivery  mechanism")  for  which  they 
have  designed  a  computer-based  user  interface  ("display")  for  accessing  the  data. 

Operating  instructions  for  an  IV  pump  (the  “content”)  may  be  in  a  paper  manual  (the 
“delivery  mechanism”). 


We  have  proposed  an  "uncoupling"  of  the  content  from  the  delivery  and  display 
mechanism  via  the  development  of  a  imiversal  bedside  information  interchange  format. 
Requiring  vendors  (manufacturers,  publishers,  hospitals,  etc.)  to  put  their  “bedside” 
information  in  a  universal  format  permits  a  common  user  interface  at  the  bedside.  The 
system  would  allow  quick  downloads  of  updated  information  from  a  variety  of  sources. 

In  this  NIH  funded  project,  we  are  developing  a  prototype  system  and  exploring  a  variety 
of  issues  including  acceptability  at  the  bedside,  administrative  concerns,  and  economic 
issues  from  the  information  providers  point  of  view.  The  project  is  complementary  to  the 
Guidelines  Interchange  Format  in  that  it  makes  other  information  useable  at  the  bedside 
as  well. 


DEVELOPMENT  AND  DISSEMINATION  OF  GUIDELINES 

Christel  Mottur-Pilson,  PhD 
Director,  Scientiflc  Policy,  ACP-ASIM 


The  electronic  medium  is  increasingly  replacing  the  usual  modes  of  clinical  knowledge 
transmission  and  dissemination,  namely  the  peer-reviewed  printed  journal.  Despite  the 
rapid  expansion  of  evidence-based  recommendations  derived  from  randomized  controlled 
trials  (RCT)  and  meta-analyses;  it  still  takes  approximately  1 3  years  before  even  half  of 
these  recommendations  are  incorporated  into  medical  practice.’  To  what  extent  this 
delay  is  a  frinction  of  knowledge  overload  or  difficulty  of  translating  research  results  into 
practice-based  application  is  an  open  question.  Yet  there  is  an  exception  to  this 
knowledge  gap;  information  available  at  the  point  of  service  is  likely  to  be  utilized  and  to 
positively  affect  the  quality  of  patient  care. 

Electronic  clinical  practice  guidelines  may  be  the  perfect  tool  for  point  of  service 
implementation.  These  electronic  guidelines  may  be  able  to  overcome  the  lengthy  delay 
between  guideline  development  and  implementation.  To  achieve  the  goal  of  rapid 
guideline  dissemination,  guidelines  have  to  be  clad  in  a  “standard”  and  generic  electronic 
format,  which  allows  local  adaptation  and  adoption.  In  essence  then,  the  success  of 
electronic  guidelines  is  measured  in  the  same  way  as  text  guidelines,  namely  through 
adaptation  and  adoption  at  the  local  or  institution  specific  level. 

To  facilitate  the  guideline  translation  process  from  text  to  electronic  format, 
guideline  authors  and  methodologists,  and  computer  scientists  have  to  share  a  basic 
understanding  of  their  respective  fields  to  facilitate  a  smooth  translation  from  text  to 

*  Antam,  Lau,  Kupelnick  et  al.  1992.  A  comparison  of  results  of  meta-analysis  of  randomized  controlled 
trials  and  recommendations  of  clinical  experts.  JAMA  268:240-248. 


electronic  representation.  In  effect,  the  constraints  of  computer  language  should  not 
hinder  the  verbal  rendering  of  guideline  recommendations  and  vice  versa. 

To  begin  the  dialogue  between  computer  expert  and  guideline  developer,  I 
propose  to  familiarize  the  electronic  experts  with  some  of  the  basic  building  blocks  that 
go  into  guideline  development.  I  plan  to  draw  on  the  Institute  of  Medicine’s  (lOM) 
seminal  work  on  guidelines.  In  particular,  I  shall  present  the  lOM  attributes  of  valid 
guidelines.  Since  the  American  College  of  Physicians-American  Society  of  Internal 
Medicine’s  (ACP-ASIM)  guideline  development  program  incorporates  these  attributes, 
description  of  the  Clinical  Efficacy  Assessment  Program  (CEAP)  will  follow. 

Since  an  interactive  question  and  answer  format  is  more  conducive  to  learning 
than  a  didactic  presentation,  I  plan  to  minimize  the  “lecture”  part,  to  encourage  open 
discussion  and  interaction.  A  moderator  ( Steven  Lascher,  DVM,  MPH)  will  keep  the 
discussion  focused. 

I  welcome  your  suggestions  and  comments. 


^  Institute  of  Medicine.  1992.  Guidelines  for  Clinical  Practice.  From  Development  to  Use.  Eds.  MJ  Field 
and  KN  Lohr.  Washington,  DC.  National  Academy  Press. 


Propositional  Representation  of  Clinical  Practice  Guidelines 
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Clinical  practice  guidelines  (CPGs)  are  aimed  at  physicians  of  various  degrees  of  knowledge  and 
experience.  The  expected  result  of  the  use  of  guidelines  is  the  standardization  and  homogenization 
of  clinical  practice  (i.e.,  that  the  same  clinieal  problem  be  diagnosed,  treated,  or  managed  similarly 
by  different  physicians).  However,  CPGs  can  be  semantically  very  complex.  In  fact,  many  CPGs 
are  eomposed  of  elaborate  collections  of  prescribed  procedures,  sometimes  involving  the 
embedding  of  procedures  within  procedures  or  complicated  temporal  or  causal  relations  among  the 
various  steps  in  a  procedure.  Furthermore,  interpretation  of  CPGs  involves  the  generation  of 
inferences  that  are  needed  for  correctly  implementing  the  procedure  prescribed  by  the  guideline. 
The  fact  that  interpretation  requires  making  inferences  makes  CPGs  highly  ambiguous  and 
therefore  prone  to  different  interpretations  by  different  physicians.  To  overcome  the  inherent 
ambiguity  of  CPGs,  physicians  make  use  of  their  domain-specific  knowledge  as  well  as  their 
general  world  knowledge.  This  knowledge  helps  disambiguate  procedural  steps  and  temporal  or 
causal  relations  among  steps.  Unfortunately,  it  is  also  the  source  of  the  differences  in  CPG 
interpretation  that  the  CPGs  are  intended  to  equalize. 

In  this  position  paper,  we  argue  that  propositional  analysis  is  an  important  tool  in  the  development 
of  CPGs.  Propositional  analysis  is  a  formal  method  for  representing  textual  information  in 
Cognitive  Science  [1].  It  allows  the  identification  of  those  aspects  of  the  guideline  that,  because  of 
their  semantic  complexity  (i.e.,  those  involving  generation  of  inferences),  may  pose  a  problem  of 
interpretation  for  the  physician.  To  this  end,  it  may  be  an  important  tool  for  the  development  of 
effective  CPGs,  when  used  in  the  design  process.  Its  utilization  as  an  alternative,  empirically-based 
form  of  representation  can  be  used  at  various  points  in  the  design  process  (e.g.,  development  of 
algorithms,  implementations  in  electronic  medical  records)  as  an  aid  to  the  development  of  the 
content  as  well  as  the  form  of  guidelines. 

Propositional  analysis  is  a  method  for  the  empirical  investigation  of  the  semantic  structure  of 
discourse,  as  in  a  collection  of  think-aloud  protocols  from  designers  and  end-users  alike.  The  basic 
unit  of  analysis  is  the  proposition.  A  proposition  is  an  idea  underlying  a  piece  of  text  (e.g.,  phrase, 
sentence,  paragraph,  etc.)  and  corresponds  to  the  basic  unit  of  the  mental  representation  of 
symbolic  information  in  human  memory.  A  proposition  is  composed  of  two  concepts  and  a  relation 
between  the  eoncepts.  For  instance,  the  sentence  “John  went  into  the  house”  expresses  one 
proposition  that  can  be  analyzed  into  the  concepts  of  “John”,  “house”,  and  the  relation  “went.” 
Propositional  analysis,  however,  does  not  end  with  the  identification  of  the  concepts  and  relations. 
It  also  involves  the  categorization  of  the  concepts  and  relations  in  the  text.  In  the  example  above, 
“John”  is  the  agent  of  the  action  and  “house”  is  a  location.  A  propositional  representation  of  the 
sentence  would  look  like  this: 

1.1  Go  AGT:  John,  LOCrhouse,  TNSrPAST; 


where  the  number  1.1  in  the  first  column  represents  the  proposition  number,  the  second  column 
“Go,”  is  the  head  element  or  argument,  and  the  third  column  is  called  the  predicate  (what  is  said  or 
predicated  on  the  argument).  Notice  that  the  propositional  representation  is  always  in  present  (use 
of  “go”  instead  of  “went”)  while  tense  information  is  given  in  the  predicate. 

Propositional  analysis  allows  us  to  identify  three  types  of  propositions  that  indicate  the  level  of 
complexity  of  the  text;  single,  embedding,  and  linking  propositions.  Single  propositions  express 
single  ideas.  These  propositions  are  self-contained;  that  is,  they  do  not  contain  or  refer  to  other 
propositions.  Embedding  propositions  contain  one  or  more  propositions  within  them.  Finally, 
Linking  propositions  connect  other  propositions  and  represent  likely  inferences.  That  is  their 
predicates  include  one  or  more  proposition  numbers.  We  argue  that  the  presence  of  embedding  and 
linking  propositions  indicates  the  greater  complexity  of  the  text.  This  means  that  as  the  number  of 
embedding  and  linking  propositions  increases  so  does  the  number  of  inferences  needed  to  interpret 
the  text.  Since  inferences  introduce  variability  in  interpretation,  a  text  with  many  of  these  types  of 
propositions  will  be  more  variable  in  its  interpretation  than  a  text  with  fewer  of  them. 

How  can  propositional  analysis  aid  in  the  development  and  utilization  of  guidelines?  Given  that 
CPGs  may  require  a  large  number  of  inferences  for  its  proper  interpretation,  the  identification  of 
embedding  and  linking  propositions  may  be  beneficial  for  the  development  of  more  explicit 
guidelines.  These  inferences  provide  coherence  to  the  representation  and  may  be  crucial  in  the 
proper  understanding  of  the  guideline.  However,  when  trying  to  represent  clinical  guidelines  in 
automated  systems  (e.g.,  Web-based,  DSS),  the  resulting  user  representation  may  fail  to  include 
information  that  is  shown  in  the  linking  or  embedding  propositions.  The  complexity  of  CPGs  can 
however  be  lessened  by  converting  complex  propositions  to  sets  of  single  propositions  easing  the 
need  for  generating  inferences  that  make  guideline  interpretation  more  variable.  To  time  the 
information  to  different  levels  of  user  expertise,  it  may  be  possible  to  make  this  information 
optional  for  browsing  in  the  CPG,  e.g.,  by  means  of  pointers.  In  this  way,  the  expert  physician  may 
skip  through  this  information  (and  therefore  make  the  necessary  inferences  himself  or  herself) 
whereas  the  inexperienced  physician  may  inspect  the  information  at  will. 
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Each  day  physicians  must  make  informed,  accurate  decisions  on  patient  care.  This  is  a  formidable 
challenge  considering  that  the  body  of  medical  knowledge  is  in  a  state  of  constant  grovvdh  and 
revision.  Clinical  practice  guidelines  (CPGs)  give  a  means  to  manage  this  ‘information  overload’ 
and  provide  up-to-date,  scientifically  valid  medical  information  to  aid  the  practitioner  in  making 
more  efficient  and  effective  clinical  decisions.  The  end  result  would  be  an  improvement  in  the 
quality  and  efficiency  of  health  care.  Although  the  outlook  for  the  potential  impact  of  CPGs  is 
positive,  reviews  and  studies  of  practice  patterns  report  that  practitioners  do  not  make  use  of  this 
support  tool  on  a  regular  basis.  This  noncompliance  by  practitioners  is  puzzling  because  CPGs  are 
meant  to  help  enhance  physician  performance.  So  why  is  there  a  breakdown?  Understanding  this 
process  in  human  behavior  can  guide  us  making  recommendations  for  improvement.  This  position 
paper  argues  about  the  importance  of  conducting  cognitive  examinations  of  the  use  of  CPGs  at  the 
“point  of  care”  in  order  to  insure  their  effective  utilization.  We  present  here  some  lessons  for  the 
development  of  CPGs  derived  from  empirical  studies  of  guideline  utilization  at  the  point  of  care 
and  how  the  findings  can  help  in  understanding  successes  and  failures  of  guideline  utilization  by 
end-users. 

We  start  by  presenting  some  fimdamental  characteristics  of  the  way  experts  and  non-experts 
interpret  and  use  information  for  making  decisions  [1].  Research  has  shown  that  medical 
practitioners  rely  on  the  heuristic  of  explanatory  sufficiency,  where  they  generate  an  explanation  for 
a  patient  problem  just  until  they  are  satisfied  that  the  explanation  is  not  too  far  off  the  mark.  That  is, 
physicians  generate  only  satisfactorily  accurate  explanations,  not  maximally  accurate  explanations. 
Since  this  is  a  fimction  of  the  level  of  expertise  of  the  physician,  the  explanations  provided  also 
vary  as  a  function  of  expertise,  which  introduces  variability  in  guideline  interpretation  and  decision 
making. 

This  difference  in  explanatory  sufficiency  is  supported  by  a  body  of  research  showing  how 
constructed  representations  fi'om  symbolic  material  vary  as  a  function  of  the  expertise  of  the  user, 
the  piupose,  and  the  nature  of  the  material  to  be  interpreted.  These  research  results,  summarized 
below,  can  provide  important  insights  into  the  design  and  development  of  guidelines  for  use  at  the 
point  of  care; 

1 .  Experts  are  more  likely  to  make  errors  of  omission,  given  the  high  level  of  abstraction  they 
employ  to  process  clinical  information.  Therefore,  for  experts,  guideline  information  can  serve 
as  reminders  that  recall  relevant  information.  For  the  non-expert,  the  situation  is  different. 

Given  their  lack  of  specialized  knowledge,  they  are  more  likely  make  error  of  commission  (e.g., 
ordering  unnecessary  tests).  Although  reminders  are  also  useful,  CPGs  help  them  focus  on  the 
relevant  information  and  discard  irrelevant  associations. 


2.  Different  purposes  require  different  guidelines.  CPGs  for  problem  solving  and  decision  making 
need  to  be  presented  in  an  easily  accessible,  and  ideally,"  just-in-time"  form  that  can  be  used  as 
part  of  the  problem  solving  or  decision  making  process.  Guidelines  for  learning  should 
encourage  reflection  and  imderstanding  the  deeper  meaning  of  clinical  situations  and  the 
reasons  for  undertaking  procedures.  Such  guidelines  can  provide  ways  to  double-check  on 
decisions  taken  and  thus  to  prevent  errors. 

3.  Different  users  may  benefit  from  guidelines  in  different  formats  (algorithms,  flowcharts,  written 
text).  Guidelines  in  diagrammatic  form  such  as  algorithms  are  useful  for  problem  solving, 
provided  that  the  user  possesses  extensive  knowledge  of  the  guideline  domain,  as  they  provide 
easy  access  to  the  relevant  information.  Algorithms  convey  only  the  major  steps  in  a  procedure 
while  leaving  room  for  physicians  to  adapt  the  guideline  to  particular  patients,  presenting  all  the 
important  information  about  a  problem  at  a  glance.  In  written  CPGs,  the  relevant  information  is 
dispersed  through  the  text.  However,  these  advantages  differ  depending  on  the  level  of  expertise 
of  the  user.  By  combining  different  forms  of  guideline  representation  a  more  effective  use  of 
CPGs  can  be  fostered. 

4.  Finally,  CPGs  should  allow  for  idiosyncrasies  of  individual  patients.  This  can  be  done  by 
supporting  expert  heuristics  that  have  been  shown  to  be  effective  in  patient  management  and  by 
introducing  considerations  regarding  the  patient’s  knowledge  and  beliefs  of  health  and  illness. 
This  requires  having  a  model  of  the  patient  as  part  of  the  decision  support  and  the  guideline,  an 
issue  that  we  are  currently  exploring. 

In  summary,  the  study  of  the  use  of  CPGs  at  the  point  of  care  is  important  because  it  provide  insights  into  how  the 
nature  and  purpose  of  guidelines  can  be  tuned  to  different  users,  when  such  research  is  used  in  the  design  process.  Thus, 
we  argue  that  this  empirical  research,  coupled  with  design  principles  from  the  cognitive  and  behavioral  sciences,  should 
be  part  of  the  information  necessary  for  the  design  and  development  of  any  form  of  clinical  guidelines. 
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With  the  advent  of  emerging  information  technologies  in  health  care,  including 
computerized  patient  record  (CPR)  systems  and  the  WWW,  possibilities  exist  for  exposing 
increasing  numbers  of  physicians  to  clinical  guidelines.  Advanced  computer  tools  have  been 
developed  for  aiding  in  the  generation  and  deployment  of  guidelines  in  electronic  form  and  promise 
to  facilitate  the  process  of  providing  guidelines  for  use  at  the  point  of  care.  Hovv^ever,  a  number  of 
fundamental  questions  remain  as  to  how  these  guidelines  can  be  best  be  deployed,  how  they  wdll 
actually  be  used  in  clinical  practice,  their  effectiveness,  and  how  their  generation  and  dissemination 
can  be  enhanced  using  advanced  computer  technology. 

In  this  position  paper  it  is  argued  that  it  is  uselul  to  apply  methods  modified  from  usability 
engineering,  called  cognitive  engineering  to  empirically  study  the  use  of  electronic  guidelines,  as 
they  are  being  developed.  Data  collected  from  such  studies  could  be  used  to  identify  problems  in 
guideline  use  and  increase  the  likelihood  that  the  information  they  provide  will  be  useful.  A  number 
of  methods  have  emerged  in  recent  years  which  have  proven  to  be  powerful  techniques  for 
analyzing  end  user  interactions  with  information  systems  as  well  as  their  informational  content 
(Nielson,  1993).  These  approaches  can  be  combined  with  methods  from  cognitive  science,  which 
focus  on  characterizing  the  cognitive  processes  involved  in  carrying  out  complex  tasks,  such  as 
interacting  with  a  computer  system.  Usability  analysis  aims  to  assess  the  efficiency  and 
effectiveness  of  information  systems  as  they  are  used  for  applications  by  representative  users,  for 
example,  the  use  of  on-line  guidelines  by  physicians  as  they  carry  out  clinical  tasks.  These  methods 
typically  involve  collecting  in-depth  process  data  on  use  of  systems,  including  audio  recording  of 
users  interacting  with  systems  or  dialogue  between  physicians  and  patients.  In  addition,  this  can  be 
supplemented  with  collection  and  analysis  of  video  recordings  of  the  actual  computer  screens  as 
physicians  interact  with  information  technologies.  The  information  obtained  from  such  study  can 
provide  valuable  feedback  into  the  iterative  re-design,  refinement  and  improvement  of  information 
systems  and  their  content. 

Guidelines  are  becoming  increasingly  available  on-line  and  physicians  are  now  able  to 
access  guidelines  while  using  a  CPR  system,  for  example,  by  browsing  through  the  system  and 
selecting  to  access  guidelines  relevant  to  a  particular  patient  encounter.  We  have  initiated  work  in 
remotely  tracking  the  use  of  guidelines  as  physicians  use  a  Web-based  CPR  system.  In  addition,  the 
technology  is  in  place  for  remotely  collecting  video  recordings  of  user  interactions  with  on-line 
guidelines.  To  date  we  have  collected  usability  data  over  the  WWW  consisting  of  records  of  user 
interactions  with  ACPOnline.  This  has  involved  presentation  of  on-line  forms  for  querying 
physician  users  at  point  of  care  as  to  why  they  are  accessing  the  guidelines  and  upon  exiting  the 
guideline  to  obtain  an  assessment  of  how  useful  the  information  was.  Our  preliminary  results 
indicate  that  physicians  use  guidelines  primarily  for  upgrading  their  general  knowledge  (typically 
while  patients  are  not  present). 

It  will  be  essential  to  extend  such  analyses  assessing  usability  of  guidelines  in  order  to 
obtain  critical  information  on  how  to  best  bring  evidence  to  bear  in  a  manner  which  is  “just  in 


time”,  and  appropriate  for  application  in  real  clinical  contexts.  Along  these  lines,  the  following  will 
be  required: 

1 .  Detailed  assessment  of  how  clinical  guidelines  are  actually  used  at  point  of  care,  using  methods 
described  above,  as  well  as  study  in  a  variety  of  work  contexts. 

2.  Identification  of  problems  or  barriers  that  reduce  the  effectiveness  or  applicability  of  guidelines, 
based  on  empirical  data  from  actual  use. 

3.  Assessment  of  the  relationship  between  computer-based  patient  record  (CPR)  technology  and 
possibilities  for  making  evidence  more  accessible  to  physicians. 

4.  Recommendations  for  improvement  in  the  uptake  of  evidence  using  guidelines,  based  on  results 
from  in-depth  usability  studies. 

In  addition,  is  argued  that  the  processes  involved  in  encoding  guidelines  must  also  be 
empirically  studied,  since  the  encoding  step  forms  the  basis  for  the  information  that  will  be 
ultimately  presented  to  end  users  online.  This  should  involve  collection  of  process  data  (e.g.  video 
and  audio  recordings)  from  subjects  interacting  with  software  for  encoding  guidelines.  Through  our 
preliminary  work  in  this  area,  we  have  been  able  to  characterize  the  steps  taken  in  modeling 
guidelines  using  the  GLIF  representation  language  and  identify  potential  difficulties  encountered  in 
the  task  of  encoding  guidelines  -  difficulties  encountered  both  with  the  underlying  guideline  and 
with  the  computer-based  representation  in  performing  this  complex  task.  This  approach  has  led  to 
identification  of  both  generic  aspects  of  the  process  of  guideline  encoding  (e.g.  complexities  in 
decision  making  involved  in  translation  of  guideline  steps  into  a  computer  algorithm)  as  well  as 
specific  problems  in  performing  this  task  (e.g.  difficulties  with  the  user  interface).  It  has  been 
documented  that  problems  related  to  user  interactions  with  systems  identified  from  study  of  few 
representative  users  are  typically  generalizable  to  much  larger  user  populations,  making  usability 
engineering  methods  cost  effective  for  providing  iterative  input  into  improving  user  interactions  and 
informational  content  [1].  These  types  of  analyses  could  be  extended  to  the  study  of  encoding  of  a 
variety  of  guidelines. 

The  information  resulting  from  in-depth  usability  analyses  can  be  fed  back  into  the  design  of 
information  contained  in  the  guidelines  and  the  fine-tuning  of  the  underlying  representational 
schemes  used  to  encode  guidelines.  This  formative  input  should  lead  to  improvement  in 
implementation  of  guidelines  and  ultimately  improve  their  effectiveness  as  they  become  deployed 
in  various  electronic  forms. 
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Guidelines  for  clinical  practice  are  being  introduced  in  an  extensive  way  in  more  and  more  different  fields  of  medicine. 
They  have  the  goal  of  indicating  the  most  appropriate  decisional  and  procedural  behavior,  optimizing  health  outcomes, 
costs  and  clinical  decisions. 

Guidelines  can  be  expressed  in  a  textual  way  as  recommendations  or  in  a  more  formal  and  rigid  way  as  protocols  or 
flow  diagrams.  In  different  contexts  they  can  be  either  a  loose  indication  for  a  preferred  set  of  choices  or  they  can  be 
considered  a  normative  set  of  rules. 

Clinical  practice  guidelines  are  seen  as  a  tool  for  improving  the  quality  and  cost-efficiency  of  care  in  an  increasingly 
complex  health  care  delivery  environment. 

Computerization  may  increase  the  effectiveness  of  both  the  information  retrieval  of  guidelines  and  the  delivery  of 
guideline-based  care.  In  an  optimal  scenario  they  are  integrated  with  the  information  systems  operational  at  the  point  of 
care.  The  full  potentialities  of  computerized  systems  can  be  exploited  in  such  an  environment  where  different  processes 
are  executed  in  parallel  on  several  patients.  In  this  context  such  systems  must  be  able  to  retrieve  the  updated  situation  of 
every  patient,  as  well  as  to  give  an  overall  report  on  the  ward,  freeing  the  physicians  to  concentrate  more  on  clinical 
decisions.  Keeping  track  of  the  parallel  activities  performed,  they  should  avoid  unnecessary  duplication  of  tasks  and 
prevent  possible  omissions. 

The  scenario  is  evolving  from  stand-alone  workstations  to  telematics  applications  that  -  utilizing  e.g.  the  Internet  (see 
for  example  http://saussure.irmkant.rm.cnr.it/guidelines  for  a  collection  of  resources)  -  not  only  support  the  use  of 
guidelines,  but  also  enable  their  development  and  dissemination.  However,  the  ever-increasing  demand  of  guidelines 
dissemination  has  to  rely  on  a  solid  conceptual  foundation  in  order  to  give  a  precise  semantics  to  the  knowledge  shared. 
Such  a  knowledge  sharing  requires  the  definition  of  formal  models  for  guidelines  representation.  The  models  should 
have  a  clear  semantics  in  order  to  avoid  ambiguities. 

The  role  of  ontologies  is  that  of  making  explicit  the  conceptualizations  behind  a  model.  In  particular  an  ontology 
contains  the  formal  description  of  the  entities  to  which  a  model  makes  a  commitment  and  of  the  relations  holding 
among  the  entities.  In  our  perspective,  an  ontology  is  a  formal  theory  which  partially  specifies  the  conceptualization 
(i.e.  the  intended  meaning)  of  the  terms  used  to  talk  about  the  entities  in  a  domain. 

The  realization  of  ontologies  is  the  groundwork  for  making  a  standard  model  acceptable  and  sharable.  An  ontology 
library  is  not  normative,  but  allows  an  inter-subjective,  explicit  and  formal  agreement  on  the  semantics  of  the  primitives 
of  a  model,  by  referring  to  more  generic  primitives  (generic  theories). 

We  defined  an  ontology  of  guidelines  which  is  part  of  a  larger  ontology  library  containing  both  domain  and  generic 
theories.  We  believe  that  such  an  approach  can  facilitate  the  standardization  process  by  allowing  an  explicit  mapping  in 
a  formal  ontology  of  the  concepts  represented  in  the  heterogeneous  models  proposed  so  far. 
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HL7  is  the  primary  interchange  standard  for  clinical  data, 
both  in  the  U.S.  and  mtemationally.  Clinical  data  sent  with 
HL7  includes  numerical,  coded  and  text  data  for  the 
Electronic  Medical  Record  (EMR.)  Besides  the  exchange 
of  actual  patient  data,  HL7  2.x  also  covers  ordering  and 
scheduling  of  clinical  work,  as  well  as  exchange  of  data 
dictionary  (master  file)  records  describing  clinical 
parameters.  HL7  2.x  order  messages  are  being  used  to 
communicate  workflow  processes  as  well  as  care  plans. 
However,  the  current  version  2.x  of  HL7  representation  of 
care  plans  is  difficult  because  of  ambiguities,  semantics 
hidden  in  syntax,  flat  structure,  unsystematic  specification 
and  implementation,  and  because  of  the  vocabulary 
problem. 

Currently  HL7  is  being  redesigned  in  its  version  3  effort, 
which  includes  the  creation  of  a  comprehensive  Reference 
Information  Model  (RIM)  [1].  The  WM  covers  all  entities 
and  data  elements  about  which  HL7  messages 
communicate.  The  HL7  RIM  went  through  multiple 
revisions  to  evolve  from  a  very  specific  reverse  engineered 
representation  of  HL7  version  2  message  content  to  a 
generalized  model  describing  most  of  the  information 
structures  of  clinical  care.  In  its  latest  revision,  the  clinical 
areas  in  the  RIM  have  changed  significantly,  to  implement 
the  Unified  Service  Action  Model  Proposal  (US  AMP) 
submitted  jointly  by  the  HL7  Committees 
Orders/Observations  and  Patient  Care  [2]  (see  appended 
Figure  1.) 

The  USAMP  restructured  the  RIM  around  a  few  clearly 
emphasized  key-concepts.  Firstly,  we  generalized  all 
clinical  events  under  the  notion  of  the  service  action.  This 
means,  no  clinical  data  exists  outside  of  a  service  action 
object.  Secondly,  we  allow  arbitrarily  complex  clinical 
expressions  using  a  typed  Service-relationship  as  a  link 
between  two  services.  Finally,  we  defined  the  concept  of 
action  mood,  to  clearly  distinguish  the  various  ways  an 
action  can  be  conceived,  i.e.  as  action  definitions,  action 
plan  instances,  and  results  of  actions  (e.g.,  observation 
results.) 

We  borrowed  the  term  action  mood  from  natural  language 
grammar  (lat.  modus  verbt).  The  notion  of  action  mood 
also  resembles  the  various  extensions  of  the  logic  of  facts 
in  modal  logic.  In  USAM,  an  action  is  represented  as  a 


predicate  A(m,  Xi,  X2, . . . ,  x„)  where  the  mood  m  specifies 
the  modality  (fact,  possibility,  intention,  goal,  etc.)  under 
which  the  truth  of  fte  predicate  is  determined.  The  other 
arguments  xj,  X2, ...,  x„  describe  actors,  subject  (patient), 
timing,  observation  value  etc. 

The  service  relationship  is  a  relation  ^Rr  where  R  is  the 
relationship  type,  5  the  source  action  and  t  the  target  action. 
Service  action  relationships  are  usually  asymmetric  and 
transitive.  Service  relationships  come  from  a  few 
categories:  generalization,  participation,  precondition, 
postcondition,  and  revision.  Generalization  allows  defining 
classical  subtypes.  The  participation  defines  sets  or 
ordered  collections  of  services,  e.g.,  the  steps  of  an  action 
plan  are  sub-services  contained  in  a  super-service. 
Preconditions  are  the  conditionals  that  allow,  suggest,  or 
prevent  an  action  from  being  executed.  Postconditions  are 
the  used  to  express  goals  (intentions)  and  maintenance  or 
end  conditions  of  actions.  Revision  is  used  to  link  action 
plan  instances  to  their  definition,  revised  action  plans  to  the 
original  plans,  and  action  events  to  action  plans. 

Because  actions  in  all  moods  are  described  through  the 
same  information  structure,  we  had  to  define  all  the  data 
elements  describing  an  action  so  that  they  would  be 
sensible  in  all  moods.  This  is  possible  if  we  conceive  all 
action  values  as  sets  that  are  successively  constrained 
across  the  different  action  moods.  For  example,  the  time  of 
an  action  in  definition  mood  is  the  set  of  all  possible  time 
periods  in  which  the  action  can  occur  (e.g.,  business  hours.) 
The  time  of  an  action  intent  is  constrained  to  a  smaller  set 
of  preferred  times.  Finally,  the  action  event  happens  at  a 
specific  time.  That  all  attributes  of  an  action  are 
successively  constrained  sets  also  explains  why  some 
observation  values  are  ranges  (e.g.,  blood  glucose  <  20 
mg/dl)  while  most  appear  to  be  points  (110  mg/dl.)  All 
measurements  are  but  constraints  of  the  actual  value  inside 
a  confidence  interval.  The  USAM  model  corresponds  well 
with  constraints  approach  to  decision  support  as  found  in 
constraint  logic  programming  (CLP)  and  related  work. 

In  the  USAM,  we  attempted  to  systematize  the  ways  HL7 
messages  communicate  the  requested  workflow  and 
completion  of  the  work.  By  means  of  the  mood  code,  we 
can  take  the  structure  that  specifies  an  action  plan  ordered 
for  a  patient  (HL7’s  primary  concern)  and  reuse  it  to 
describe  general  action  plans,  knovra  as  care  paths  or 
guidelines. 


Likewise,  the  US  AM’s  reuse  of  one  action  structure  in 
different  mood  guarantees  that  any  clinical  data  that  is 
reported  can  also  be  used  in  guidelines  as  conditional 
criteria  or  goals.  Since  most  clinical  data  are  reportable 
through  HL7,  we  can  directly  use  this  data  in  guidelines. 
This  overcomes  the  impedance  mismatch  between 
guideline  systems  and  the  medical  record  that  is  evident  in 
the  Arden’s  “curly  braces  problem”  and  in  the  unspecified 
conditional  expressions  in  GLIF. 

We  found  many  similarities  between  the  US  AM  and  other 
guideline  work,  notably  the  notion  of  skeletal  plans  in 
ASBRU  [3],  and  the  model  based  approach  to  guidelines 
defined  in  EON  [4]  and  GLIF  [5].  The  USAM  group  now 
seeks  to  cooperate  with  the  HL7  Clinical  Decision  Support 
TC  (Arden)  and  the  InterMed  group  to  refine  our  model- 
based  approach  to  guidelines. 

We  believe  that  such  a  cooperation  could  be  of  benefit  to 
all  parties  involved.  The  HL7  guideline  features  could 
grow  through  the  Expertise  in  the  InterMed  project  to  a 
robust  technology  that  is  actually  tested  and  used  in  current 
and  new  guideline  systems.  On  the  other  hand,  the 
InterMed  project  could  benefit  from  being  embedded  into 


HL7’s  linkage  to  the  medical  record,  and  its  ability  to  move 
standard  messages  into  public  use.  Together  we  could 
make  truly  interoperable  guideline  exchange  work. 
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Figure  1:  This  is  the  complete  class  diagram  of  the  USAMP-II  model,  covering  the  clinical  and  ancillary  part  of  the 
entire  HL7  RIM.  This  includes  the  traditional  RIM  areas:  orders,  service  event,  master  service,  scheduling,  and 
patient  care.  The  three  service  related  class  hierarchies  (formally  called  master  service,  service  order,  and  service 
event  have  been  merged  into  one  Service  hierarchy.  The  attribute  “mood_cd”  distinguished  between  different 
nuances  of  a  service  (e.g.,  whether  it’s  the  service  as  ordered,  as  scheduled,  as  performed,  as  reported  in  history 
taking,  as  a  goal,  etc.)  The  second  important  novelty  is  the  unification  of  Material  that  includes  all  the  substantial 
“things”  (except  people)  that  sen/ices  deal  with.  In  spite  the  dramatic  decrease  in  attributes,  all  current  application 
layer  requirements  of  HL7  version  2.3.1  are  covered. 
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Objective:  Our  long-term  goal  is  to  apply  the  knowledge  contained  in  practice  guidelines  to  support 
clinical  decision-making.  As  a  requisite  step  toward  that  goal,  we  have  designed  a  model  using  XML  to 
simplify  and  standardize  encoding  of  the  content  of  guideline  documents.  The  model  includes  a  sufficiently 
broad  set  of  concepts  to  be  useful  throughout  the  guideline  lifecycle. 

Design;  Current  guideline  document  models  are  limited  in  that  they  often  reflect  only  the  specific 
orientation  of  their  designers.  Thus,  developers  and  disseminators  provide  few  constructs  in  their  models 
for  conceptualizing  guideline  recommendations,  while  implementers  de-emphasize  concepts  that  are 
necessary  to  establish  a  guideline’s  validity.  We  developed  the  Guideline  Elements  Model  (GEM)  using 
XML  to  represent  the  heterogeneous  knowledge  contained  in  practice  guidelines.  Core  constructs  were 
derived  from  the  Institute  of  Medicine’s  Guideline  Appraisal  Instrument,  the  National  Guidelines 
Clearinghouse,  the  augmented  decision  table  guideline  representation,  and  the  Guidelines  Interchange 
Format  (GLIF).  These  were  supplemented  with  additional  concepts  derived  from  a  literature  review. 

Results:  The  GEM  hierarchy  includes  more  than  100  elements.  Major  concepts  relate  to  a  guideline’s 
Identity,  Developer,  Purpose,  Intended  Audience,  Method  of  Development,  Testing,  Review  Plan,  and 
Knowledge  Components.  Knowledge  Components,  in  turn,  include  Recommendations  (which  comprise 
Conditionals  and  Imperatives),  Eligibility  Criteria,  Definitions,  and  Algorithms.  The  GEM  Document  Type 
Definition  and  Schema  can  be  used  to  validate  guideline  documents  represented  as  GEM  files.  We  have 
marked  up  several  guidelines  from  a  variety  of  sources  to  establish  the  comprehensiveness  of  the  GEM 
ontology. 


Conclusion:  GEM  is  expressively  adequate  to  represent  the  heterogeneous 
information  contained  in  guidelines.  Use  of  XML  contributes  to  a  flexible, 
comprehensible,  sharable,  and  reusable  knowledge  representation  that  is 
both  human-readable  and  computable. 


Current  Activities:  We  are  creating  tools  to  assist  with  guideline  markup 
and  automated  implementation  of  guideline  recommendations.  The  proposed 
model  is  being  evaluated  in  informatics  laboratories  at  2  other  institutions  to 
investigate  differences  in  the  organization  of  medical  concepts  when  GEM  is 
used  to  represent  guideline  knowledge. 


Clinical  Practice  Guidelines:  Principles  for  Life  Cycle  Systems 
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We  envision  a  world  with  tens  of  1,000s  of  guideline  rules  (it  took  us  1,000  rules 
just  to  make  the  CDC’s  vaccine  guidelines  executable)  -  too  many  for  each  organization 
to  re-program.  So,  of  primary  importance  is  the  scalability,  reliability,  adaptability,  and 
portability/seemless  integration  of  large  collections  of  executable  guidelines.  To  support 
this  view,  we  have  been  constructing  a  specification  for  an  open  software-based,  formal 
method  for  engineering  of  machine-executable  practice  guidelines  into  decision  support 
environments.  Called  Computer  Aided  Decision  Support  Engineering  (CADSE),  it  is  a 
top  level  map  of  what  life  cycle  engineering  should  encompass  as  pertains  to  clinical 
practice  guidelines,  caremaps,  trial  eligibility  rules,  and  the  like. 

What  is  needed  is  an  environment  that  utilizes  software  engineering  tools,  formal 
methods,  and  open  standards  to  permit  cross-organization  authoring  and  local  tailoring  of 
guidelines  with  appropriate  management  of  versions,  copyrights  and  ownership,  and  that 
provides  public  interfaces  that  permit  any  vendor's  decision  support  tools  to  display  the 
guidelines  within  their  proprietary  interfaces.  Specifically,  we  follow  the  Embedded 
Systems  Principle:  executable  guidelines  for  decision  support,  protocols  for  clinical 
trials,  and  so  on  must  interact  with  clinicians  and  patients  fiirough  current  vendors’ 
applications  (e.g.,  patient  record  systems,  charting,  order  entry,  clinical  trial  software, 
results  reporting  applications,  etc.)  -  the  so-called  “legacy”  software. 

We  also  envision  a  distributed  set  of  clinical  specialists  who  wish  to  create  new 
guidelines  and  retrieve  and  adapt  them  for  local  practice.  At  present  this  community 
exists  as  authors,  adaptors,  and  maintainers  of  1,000s  of  non-executable,  electronic 
guideline  documents  many  of  which  are  on  the  web. 

In  between  the  authors  and  the  vendor  applications  is  the  role  for  CADSE,  which  itself 
may  be  thought  of  as  a  three-layered  block.  At  the  top  layer,  there  needs  to  be  an 
environment  to  support  the  authoring,  checking,  local  adapting  and  updating,  version 
management,  and  maintenance  of  guidelines.  Authoring  and  Local  Adapting  Principle: 
This  environment  should  collect  and  display  guideline  knowledge  and  other  forms  of 
evidence  via  the  help  of  models  of  generic  tasks  in  guideline  authoring,  skeletal  plan 
refinement,  terminology-enabled  elicitation,  visual  programming,  and  wizard  assistance 
and  critiquing.  Central  to  this  idea  are  the  semantics  of  not  just  guideline  conditions  and 
steps  (current  GLIF)  but  also  of  higher  level  artifacts  (e.g.,  temporalities  and  caremaps, 
eligibility  questionnaires  and  uncertainties,  and  treatment  tradeoff  tables,  among  many 
others)  that  can  only  be  discovered  as  one  begins  to  construct  visual  programming 
widgets  that  are  the  intuitive  forms  in  which  clinicians  think  and  v^dll  seek  to  actually 
author,  adapt,  and  update  guidelines.  CADSE  also  requires  the  guideline  knowledge  to  be 
unambiguously  terminology-tagged  and  to  be  represented  in  an  open  standards, 
programming  language-neutral,  canonical  form  (CADSE  middle  layer)  for  which  parsers 


can  readily  be  deployed  to  translate  it  to  any  vendor’s  format,  although  vendors  will  have 
to  support  such  interfaces. 

Equally  vital  is  for  CADSE  to  provide  formal  methods  for  ensuring  intra-  and 
inter-guideline  reliability.  Clinical  guidelines  are  complex  objects,  and  guideline 
authoring  is  an  error-prone  process.  Since  patient’s  health,  and  even  life,  may  be  at  risk,  it 
is  critical  that  one  assures  the  semantic  consistency  of  guidelines.  Reliability 
Engineering  Principle:  In  general,  a  guideline  (or  a  set  of  guidelines)  is  inconsistent  if  it 
can  recommend  contradictory  actions,  which  should  never  be  taken  together,  or  fails  to 
give  a  recommendation  when  one  is  needed.  Process  algebra  could  help  us  find  and 
eliminate  inconsistencies  if  a  few  changes  are  made  to  GLIF.  GLIF  guidelines  have  a 
fixed  declarative  structure,  that  is:  1)  conditions  and  eligibility  criteria  are  clearly 
separated  from  actions;  2)  information  flow  through  the  guideline  is  fixed  and  can  easily 
be  traced  by  a  computer  program.  The  only  difficulty  is  that  GLIF  was  originally  created 
as  a  structural  semantics  based  on  CORBA’s  Interface  Definition  Language  or  IDL.  So 
for  reliability  checking  formalisms  to  work,  one  must  have  a  process  (state  transition) 
view  of  the  semantics.  One  must  extend  GLIF’s  semantics  for  criterion  logic  based  on 
process  and  temporal  algebra.  At  present  the  GLIF  has  neither  primitives  for  building 
step  (e.g.,  criteria)  statements  nor  terminology  standards,  but  instead  allows  free  text 
statements  for  all  steps.  This  makes  it  difficult  to  manage  terminology,  to  verify  logical 
actions,  and  to  handle  data  and  actions.  What  is  needed  is  to  add  to  the  GLIF  standard  a 
terminology-enabled  conditional  logic  for  representing  statements  such  as  criteria, 
conditionals,  If-Then-Else  type  of  sentences,  and  so  on.  With  these  simple  extensions 
added  to  GLIF,  we  can  apply  process-algebraic  techniques  to  guideline  analysis. 

As  mentioned  above,  the  middle  layer  of  CADSE  expects  the  executable  guideline  to 
be  represented  in  a  language-neutral  canonical  form.  This  applies  to  the  semantics  not 
just  of  the  knowledge  model,  but  also  of  the  information  and  data  model  implicit  in  every 
guideline  document  and  that  is  vital  to  the  ultimate  binding  and  integration  of  that 
guideline  with  patient  data.  At  present,  however,  GLIF  ignores  the  information  and  data 
model.  Terminology  tagging  of  guideline  lexicon  is  an  essential  step  to  alleviating  this 
difficulty.  However,  that  is  only  half  the  battle  since  guidelines  often  require  a  number  of 
intermediate  operations  to  be  performed  to  patient  data  before  they  can  use  it  (e.g., 
aggregation,  averaging  of  observations,  trend  analysis,  filtering,  translation,  units 
conversion,  etc.,  etc.).  We  call  these  intermediate  operations  “knowledge  mediation 
steps”.  Knowledge  Mediation  Principle:  For  guidelines  to  become  truly  portable,  any 
canonical  representation  must  terminology  tag  both  the  knowledge  and  information 
models  of  executable  guidelines,  and  require  vendor  applications  to  map  their  local 
dialects  to  the  standardized  terminology  set.  Further,  it  is  vital  to  adopt  standards  (e.g.. 
Object  Query  Language  and  Object  Definition  Language)  and  toolsets  that  allow  the 
rapid  authoring,  adapting,  and  sharing  of  reusable  libraries  of  mediators  that  support  the 
intermediate  operations.  At  present  GLIF  ignores  both  pieces  of  this  principle  -  the 
information  model  and  the  mediation  code. 

Finally,  the  lowest  layer  of  a  CADSE  makes  guideline  knowledge  a  resource  and 
service  the  vendors’  applications  obtain  from  the  operating  system,  much  as  they  obtain 
other  execution  level  services  (e.g.,  event  handling,  publish  and  subscribe  brokers,  name 


servers,  encryption,  message  routing,  etc.).  Some  of  these  services  are  germane  only  to 
medical  applications  such  as  terminology  service  and  master  patient  index.  Guideline 
Execution  Services  Principle:  Still  other  of  these  services  are  peculiar  to  guidelines 
alone,  such  as  a  virtual  guideline  machine  (execution  engine),  a  mediator  generator  and 
wrapper,  and  a  matchmaker  -  a  broker  that  polls  for  the  proper  guidelines  to  run.  It  is 
vital  to  create  and  implement  such  services,  if  CADSE  is  to  become  a  reality. 
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We  address  a  problem  of  authoring  and  maintaining  large  collections  of  clinical 
guidelines.  The  process  of  guideline  authoring  is  a  difficult  and  error-prone  activity.  At 
the  same  time,  guidelines  are  safety-critical  systems,  because  health  and  even  life  of 
patients  depends  on  correctness  of  procedures  prescribed  by  the  guidelines.  Because  of 
this,  it  is  extremely  important  to  verify  guidelines  before  they  are  used. 

We  propose  an  approach  to  analyze  consistency  of  guidelines  by  means  of  a 
mathematical  formalism  known  in  computer  science  as  formal  methods.  Formal  methods 
are  used  for  analysis  of  safety-critical  systems  in  avionics,  aerospace,  telecommunication 
industries,  etc.  Formal  methods  treat  a  system  under  consideration  as  a  mathematical 
object,  and  provide  a  proof  that  the  system  complies  with  its  requirements.  The  proposed 
validation  approach  will  be  a  part  of  a  larger  environment  for  authoring  and  maintenance 
of  clinical  guidelines  called  CADSE  (Computer  Aided  Decision  Support  Engineering)  that 
is  being  developed  at  the  University  of  Pennsylvania. 

The  large  amount  of  data  that  is  involved  in  guideline  analysis  makes  it  a  daunting 
task.  Early  work  on  semantic  analysis  of  rule-based  systems  [1,2]  failed  to  yield  practical 
results.  In  these  systems,  analysis  was  made  intractable  by  very  expressive  languages  used 
for  expressing  rules.  These  languages  allow  the  rules  interact  in  arbitrary  ways,  making 
exhaustive  analysis  of  all  possible  interactions  infeasible  or  even  impossible. 

If  we  want  to  thoroughly  analyze  thousands  of  guidelines,  we  need  a  more 
structured  language  for  expressing  the  guideline  rules.  An  ideal  language  would,  on  the 
one  hand,  allow  guideline  authors  to  express  everything  they  need  in  a  guideline;  on  the 
other  hand,  the  language  will  promote  a  disciplined  way  of  writing  rules,  keeping  analysis 
complexity  low.  The  GLIF  language  [3]  has  a  possibility  of  being  used  as  such  structured 
language.  The  advantage  of  GLIF  is  that  it  organizes  the  rules  in  the  form  of  a  graph, 
representing  decisions  made,  and  actions  taken,  during  the  execution  of  a  guideline.  TTiis 
representation  provides  a  clear  separation  between  conditional  rules  and  eligibility  criteria 
from  actions  recommended  by  the  guideline.  In  addition,  the  structure  of  a  graph 
explicitly  represents  the  order  of  application  of  guideline  rules.  These  two  provisions 
make  analysis  of  the  guideline  much  easier. 

We  consider  three  types  of  inconsistencies  in  guidelines  that  can  be  analyzed  by 
means  of  formal  methods: 

•  Intra-guideline  consistency  analysis  deals  v\dth  inconsistencies  vvithin  a  single 
guideline.  Intra-guideline  inconsistencies  include  struetural  defects,  such  as 
circular  paths  and  dead-end  branches;  ambiguities,  when  a  guideline  may 
recommend  more  than  one  action  for  the  same  combination  of  input  data; 
incomplete  coverage,  when  no  action  is  recommended  for  a  certain  input. 

•  Inter-guideline  consistency  analysis  is  concerned  with  detecting  interference 
between  guidelines  for  separate  medical  conditions.  When  more  than  one 
guideline  is  applicable  to  a  patient,  independently  developed  guidelines  may 
recommend  conflicting  actions.  For  example,  drugs  that  caimot  be  taken 


together  may  be  prescribed  by  different  guidelines.  It  is  important  that  the 
possibilities  of  such  interference  are  detected  before  the  guidelines  are 
deployed. 

•  Inter-author  consistency  analysis  ensures  that  versions  of  the  same  guideline, 
developed  by  different  authors  recommend  the  same  actions  under  all 
circumstances.  Independent  development  of  a  guideline  by  several  authors  is 
often  used  to  avoid  oversights  of  a  single  author.  Comparison  of  the  resulting 
guidelines  is  a  non-trivial  task  that  is  itself  prone  to  errors.  A  mechanical 
support  for  such  comparison  is  an  important  feature  of  a  guideline  authoring 
system. 

In  order  to  fully  support  analysis  of  the  three  types  of  guideline  inconsistencies, 
semantics  of  GLIF  constructs  need  to  be  refined  and  extended.  Currently,  conditional 
expressions  and  actions  are  stored  as  textual  attributes  of  conditional  and  action  nodes, 
respectively.  To  carry  out  the  analysis,  the  language  of  conditions  and  actions  need  to  be 
fixed  and  its  semantics  precisely  defined. 

To  summarize,  our  future  work  on  guideline  analysis  will  be  carried  out  within  the 
CADSE  development  project,  a  framework  for  computer-aided  guideline  authoring  and 
maintenance.  Our  two  main  goals  with  respect  to  computerized  analysis  of  guidelines  are: 
1)  provide  extensions  to  GLIF  semantics  that  would  enable  formal  analysis;  2)  adapt 
existing  formal  methods  to  the  specific  needs  of  guideline  analysis. 
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Position  statements 

Premise  -  Despite  the  great  emphasis  that  medical  community  put  on  clinical  practice 
guidelines  (GLs)  during  last  years,  several  problems  are  associated  with  GLs  diffusion 
and  implementation  in  clinical  setting.  “Official”  GLs,  delivered  by  health  care 
authorities  (health  agencies,  medical  associations,  health  policy  makers,  etc),  may  show 
two  opposite  faults:  if  they  are  very  general  their  interpretation  is  difficult  and 
ambiguous,  while  if  they  are  very  specific,  they  may  not  fit  the  organisational  constraints 
of  the  different  environments  where  they  must  be  implemented.  In  addition,  and 
independently  from  organisational  reasons,  health  care  operators  often  do  not  fully 
comply  with  GLs,  simply  because  it  is  difficult  to  impose  behavioural  changes.  If  a  local 
standard  already  exists,  physicians  offer  resistance  to  change,  and  this  is  reasonable, 
especially  if  they  do  not  perceive  an  advantage.  To  this  concern,  it  must  be  recognised 
that  often  physicians  are  not  provided  with  instruments  for  evaluating  the  benefits  of  a 
GL  introduction,  thus  lacking  the  opportunity  to  be  convinced  by  the  evidence  of 
improved  outcomes.  Last  but  not  least,  guidelines  are  often  represented  as  a  set  of  rules, 
and  little  emphasis  is  put  on  decisions  requiring  a  “utility”  assessment. 

This  premise  motivates  our  current  efforts  in  the  field  of  guideline  integration  into  the 
clinical  practice.  In  the  following  our  current  position  is  summarized: 

-  We  developed  a  formalism  for  computerised  GL  representation.  Different  formalisms 
have  been  developed  by  other  groups  and  we  agree  that  efforts  must  be  done  in  order 
to  create  a  common  language  for  knowledge  sharing.  We  also  should  promote  the  use 
of  these  tools  by  the  experts  developing  GLs,  because  often  the  prose  is  ambiguous 
and  the  "textual"  GL  is  not  complete.  On  the  contrary,  a  computerised  representation 
requires  (or,  at  least,  is  able  to  verify)  the  GL  logical  correctness; 

We  are  working  on  the  “compliance  problem”.  We  know  that  it  may  be  improved  by 
integrating  GL  with  the  electronic  patient  record  (EPR);  to  this  concern,  it  must  be 
stressed  that  for  performing  sound  statistics  on  the  GL  impact  it  is  necessary  to  verify 
which  tasks  have  been  completed  and  the  motivations  for  the  possible  non- 
compliance;  integrating  GL  with  EPR  implies  tackling  the  terminology  problem  as 
well; 

We  think  that  different  inference  engines  (i.e.  mechanisms  for  advice  production, 
given  the  specific  patient  data)  must  be  allowed  for  the  same  GL,  in  order  to  achieve 
a  user  interaction  that  is  tailored  to  the  specific  clinical  setting.  As  for  any  other 
information  tool  or  decision  support  system,  it  is  very  important  that  the  GL  do  not 


badly  interfere  with  the  clinical  routine.  A  user  needs  analysis,  as  well  as  a  workflow 
analysis  of  the  clinical  environment  must  precede  the  GL  implementation. 

-  The  computerised  representation  must  also  facilitate  the  GL  evaluation.  Following  the 
most  recent  medical  informatics  commimity  suggestions,  the  representation  must 
embed  knowledge  about  the  goal  of  the  guideline  and,  when  it  is  the  case,  about  the 
goal  of  particular  tasks  composing  the  guideline  itself  Goal-related  outcomes  are  then 
stored  in  the  EPR,  in  such  a  way  to  measure  the  GL  benefit. 

We  focus  the  attention  on  the  GL/user  interaction,  and  we  distinguished  three  key 
interaction  aspects  that,  in  general,  may  be  tackled  differently,  according  to  the  specific 
user  needs. 

1.  The  advice  production:  it  can  be  “explicit”,  i.e.  as  long  as  a  task  is  performed,  the  GL 
suggests  the  next  steps.  This  can  be  a  suitable  modality  for  begiimers.  On  the 
contrary,  it  can  be  “silenf’,  i.e.  the  user  is  advised  only  when  a  non-compliance  is 
detected.  This  can  be  worth  for  expert  users. 

2.  The  approach  to  non-compliance:  a  GL  could  proceed  to  the  next  task(s)  only  if  the 
user  fully  complied  with  the  GL  for  what  concerns  the  previous  tasks.  This  should  be 
the  case  for  particularly  mandatory  procedures;  another  possibility  is  that  the  GL 
proceeds  also  in  case  of  non-compliance,  but  only  if  the  user  provides  a  justification 
for  that;  finally  the  GL  could  proceed  in  any  case,  just  storing  the  non-compliance. 

3.  The  intervention  time:  a  GL  could  react  in  real  time  to  the  users  actions,  and  suggest, 
in  real  time  as  well,  the  next  action(s).  This  implies  for  the  user  the  possibility  of 
interacting  very  frequently  with  the  computer  and  thus  a  very  efficient  and  distributed 
information  system  is  necessary;  on  the  contrary,  the  user  could  access  the  GL  only  in 
specific  time  slices. 

These  three  aspects  concern  the  real-world  implementation  of  the  GL,  but  another 
important  issue  is  the  GL  use  for  educational  purposes,  where  the  focus  is  on  aspects  such 
as  explanations  and  literature  pointers,  and  where  the  user  simulates  patients,  by  filling  a 
fictitious  patient  record  with  clinical  findings,  and  he/she  v^dll  obtain  advises. 

From  Guidelines  to  CAREFLOW  management  system 

Which  is  the  technological  solution  for  the  above  mentioned  application  problems? 
Modelling  medical  knowledge  into  a  guideline  allows  establishing  what  to  do,  but  it  is 
necessary  as  well  to  model  organisational  knowledge  because  it  allows  to  establish  how 
and  hy  whom  to  do.  Our  opinion  is  that  a  Workflow  Management  System  (WFMS)  could 
be  the  eorrect  tool  to  fiilly  implement  a  GL  and  to  control  its  outcomes.  In  fact,  a  WFMS 
is  defined  as  “a  system  that  completely  defines,  manages,  and  executes  workflow 
processes  through  execution  of  software  whose  order  of  execution  is  driven  by  a 
computer  representation  of  the  workflow  process  logic”.  When  the  medieal  process 
model  is  provided  by  a  GL,  we  refer  to  the  system  as  a  GL-based  WFMS  or, 
alternatively,  CAREFLOW.  Through  such  a  system,  we  could  be  able  to  answer 
questions  as  Is  there  any  bottleneck  in  the  hospital  structure  that  impairs  the  GL 
implementation?,  How  much  does  it  cost  to  implement  the  GL  ?,  Is  any  human  or 
technological  resource  over-  or  underloaded?,  and  so  on 


An  important  aspect  to  take  into  consideration,  to  save  time  and  resources  from  the 
development  point  of  view,  is  that  WFMS  are  very  common  in  real  world  settings  other 
than  health  care.  Thus,  we  tried  to  exploit  results  achieved  in  those  contexts,  by  importing 
the  sharable  technology  into  the  health  care  context.  In  other  words,  we  propose  a 
methodology  for  integrating  research  tools  developed  in  our  laboratories  with  available 
commercial  tools  able  to  manage  classical  workflow  models. 

As  a  bench-test  we  experimented  the  implementation  of  a  GL  for  the  stroke  management, 
actually  under  evaluation  in  four  Italian  Stroke  Units  (the  GLADIS  Italian  group  - 
GuideLine  Application  for  Decision  in  Ischemic  Stroke).  This  project  aims  at  evaluating 
the  benefit  of  GLs  for  the  management  of  a  disease  that  represents  the  third  cause  of 
death  in  industrial  countries,  and  the  first  cause  of  permanent  disability.  It  is  then  a  source 
of  both  direct  and  indirect  social  costs.  Preliminary  data  on  about  400  patients  show  that 
the  guideline  application  improves  outcomes  and  decreases  costs. 
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Health  Care  Telematics  Applications  Programme.  The  authors  thank  S.K.  Andersen  and 
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Integrating  guidelines  into  the  routine  workflow  of  patient  care  has  been  a  critical  obstacle  to 
effective  use  of  any  guideline.  Most  of  the  clinical  reminders  generated  by  systems  in  operation 
invoke  simple  rules  (e.g.,  health  maintenance  reminders,  drug  interaction  checking,  disease- 
specific  treatment  recommendations).  Guidelines  that  are  more  complicated  or  involve  more 
professional  interpretation  are  often  difficult  to  implement  in  computer  programs  because  they 
require  precise  definitions  and  complete  data.  Until  executable  guidelines  tailored  to  specific 
patients  become  available  on  a  routine  basis,  healthcare  professionals  will  need  to  access,  read,  and 
interpret  textual  guidelines  themselves.  Such  textual  guidelines  constitute  the  preponderance  of 
guidelines  currently  available.  Finding  an  effective  way  to  implement  these  guidelines  (many  of 
which  are  available  as  HTML  documents  on  the  Internet  or  Intranet)  would  leverage  a  substantial 
body  of  clinical  practice  knowledge  that  is  currently  underused. 

We  propose  a  mechanism  to  integrate  textual  guidelines  at  the  point  of  care  using  electronic 
medical  records  systems.  Initially,  the  textual  guidelines  would  be  published  guidelines  as  they 
appear  in  print  or  on  Internet  Web  sites.  Later,  when  more  sophisticated  inferencing  methods  are 
employed  to  apply  domain  knowledge  to  specific  patient  contexts,  we  propose  that  the  output 
recommendations  also  be  returned  to  the  EMR  as  textual  documents,  which  can  be  processed  using 
the  same  mechanism  used  by  the  static  documents. 

We  propose  that  an  XML  document  be  defined  that  includes  provisions  for  embedding 
standardized  language  that  allows  an  EMR  system  to  process  the  user-selected  actions  fi'om  the 
guideline  as  EMR  orders,  thereby  eliminating  any  redundant  action  required  to  "transcribe"  a 
guideline  recommendation  into  an  EMR  order.  The  XML  document  could  be  composed  of  a  static 
document  published  by  a  guideline  development  organization  or  a  dynamically  generated  set  of 
recommendations  derived  from  a  decision  support  system.  Agreement  on  the  format  and  syntax  of 
the  EMR-enabled  XML  guideline  document  would  be  a  simple  method  to  ensure  compatibility 
among  guideline  developers  and  EMR  vendors. 

We  envision  that  the  next  step  in  guideline  integration  would  be  for  a  guideline  server  to  provide 
patient-tailored  recommendations  to  the  CPR  without  requiring  users  to  read  through  the  guideline 
themselves.  This  would  require  more  sophisticated  structure  in  the  guidelines  and  the  ability  for 
the  guideline  server  to  perform  reasoning.  One  possible  scenario  for  the  interaction  between  the 
guideline  server  and  the  CPR  system  could  be  the  following.  The  CPR  would  send  a  message  to 
the  guideline  server  requesting  the  invocation  of  a  specific  guideline.  The  guideline  server  would 
return  a  query  back  to  the  CPR  requesting  specific  information  about  the  patient  needed  to  custom- 
tailor  the  recommendations.  This  request  could  take  the  form  of  an  XML  document  (compliant 
with  HL7  PRA  standards)  with  certain  fields  presented  as  blanks  (e.g.,  query  by  example).  The 
CPR  would  then  fill  in  the  required  fields  and  pass  back  the  XML  document  containing  patient 
data  to  the  guideline  server.  The  guideline  server  would  process  the  patient  information  in  the 
context  of  the  relevant  guideline  and  return  a  patient-specific  recommendation  to  the  CPR  for 
presentation  to  the  user  in  the  form  of  an  XML  document. 


Appendix  C 

Summary  presentations  from  breakout  groups 


The  Four  P’s 

GLIF  Conference: 

Functional  Requirements  Break- 
out  Group 

©Problems  to  be  addressed 

•  Prioritize 

•  Plan 

Dan  Kent  &  Perry  Miller,  Recorders 

•  People 
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Problems  to  Consider 

More  Problems  of  Concern 

•  Boundaries  of  GLs 

•  Handle  uncertainties  in  evidence 

-  what  is  GL,  an  API,  a  translator 

•  Tailoring  issues,  how  flexible? 

•  Access  to  input  data 

-  mediate  data  translators  (e.g.  DOB  to  age) 

•  Coupling  GL  to  work  flow 

•  Maintenance 

•  Handle  negative  recommendations 

•  Handle  patient  preferences 

•  Consider  costs  and  quality  of  operators 

•  Unambiguous  terminology  tags 

•  Supports  abstractions  in  constructs 

•  Outputs  represented  as  actions 

And  Further  Concerns 

•  Verifiability 

•  Reliability  of  Replication 

•  Human  Comprehension  limits 

-  Readability,  expressivity  controls 

•  Present  clear  statements  of 

-  strength  of  evidence 

-  strength  of  recommendations 

-  magnitude  of  anticipated  benefits 


Boundaries  for  GLs 

•  What  to  specify  as  within  GL,  versus  what 
makes  linkage  of  GL  easier? 


Access  to  Input  Data 


Basic  access,  data  ports  or  user  entry 
Semantic  matches 

-  blood  sugar  must  equal  glucose 

-  mg/dL  =  mM/L  conversion? 

Coping  strategies  for  missing  data 

-  in  design  versus  in  real  time? 

-  Does  GL  stop,  proceed  or  seek  alternate  source? 


From  GL  into  Clinical 
Workflow 

mechanism  or  several  to  do  this  are 
essential 

GL  as  text,  hard  to  translate  into  work 
In  text,  no  more  than  a  few  ideas  and  <1 
page 

-  the  greasy  plastic  laminate  placed  in  each  chart 

-  one  click  drill-down  is  one  too  many 


Maintenance  mechanisms 


content  &  technical  maintenance 
update  management 

-  timely,  accurate 

-  responsive  to  recalls 
interactive  with  local  adaptations 

Technical  tools  vs.  Governance  &  Backing 
for  maintanence? 


Handle  Negative 


Recommendations 


some  tests  are  explicitly  identified  as 
useless  by  GL 

some  RX’s  are  contraindicated 

proper  timing  of  a  test  precludes  testing  too 

early  or  too  late 


Handle  Wants  &  Uncertainties 


Represent  the  Evidence 


•  Have  a  representation  for  patient  inputs, 
especially  their  preferences. 

•  External  to  GL,  could  be  utility,  QALE 

•  Have  a  representation  for  evidence 
uncertainties,  especially  when  writers 
advise  “present  patient  and  let  them  decide” 
-  What  to  do  when  the  GL  says:  at  this  juncture, 

be  sure  to  liilly  counsel  the  patient  that  we 
don’t  know  what  to  do  for  him  or  her. 


Strength  of  evidence 
Estimated  magnitude  of  benefit 
Strength  of  recommendation 

Handled  well  by  USPHSTF  and  by  ACP- 
ASIM  in  MRI  for  Neuro-imaging  papers 


Tailoring  GL  Issues 

Mechanism  for  handling  common  local 
variances 

-  cheaper  local  services  obviates  or  flips  a  cost 
consideration 

-  some  services  not  available  (Should  the  GL 
accommodate  or  should  the  practitioner  mobilize  to 
acquire  the  more  efficient  resource?) 

How  to  handle  the  updating  of  a  GL,  when 
the  GL  has  been  modified  by  several  users 
or  purchasers? 


Terminology  «&  Language 

•  unambiguous  terminology  tags 

•  comprehensible  representations 

-  readability,  control  of  expressivity 

*  can  show  text,  visual  flow,  programming  code 

-  support  absstractions 

*  GLIF  nested  subguidelines 

*  abstracts  that  have  detailed  specs,  e.g.  what  is  a 
drug? 


Costs  and  Quality  of  Services 

•  Mechanism  to  include  very  desirable 

•  Costs  of  tests  when  diagnostics  are  equal 

•  Costs  of  RX  when  outcomes  are  equal 

•  Quality  of  services  rendered  must  be  up  to 
performance  standards 

-  e.g.,  quality  of  high  volume  hospitals  for  surgeries 

-  neurosurgery  for  carotids  must  include  low  peri-op  stroke  rate 


Work  Plan  for  Discussion 

•  Use  this  conference  list  as  seed 

•  Generate  more  candidate  requirements 

•  Add  to  the  contributions  by  reviewing 
existing  GL  implementation  efforts 

-  MC,  I>uke,  GHC,  Kaiser,  more  active  health  plans,  others 

•  Refine  by  setting  boundaries  &  priorities 

•  Write  draft  of  “specifications  clarification” 

•  other  steps? 


Heterogeneous  Networks 

•  The  IPA  and  the  medical  group 

•  Single  large  site,  common  platform 

•  Multi-site,  common  platform 


•  Multi-site,  multi-platform 


The  Four  P’s 

GLIF  Conference: 

Functional  Requirements 

•  Problems  to  be  addressed 

Break-out  #2 

•  Prioritize  -  Its  all  important! 

•  Plan  -  Nike  sez:  Just  Do  It! 

Dan  Kent  &  Perry  Miller,  Recorders 

•  The  People  say  ? 

Boston,  MA 

March  3-4,  2000 

Additions  from  Day  2 

•  Transportable  among  collaborators 

•  From  Level  A  or  B  in  GLIF,  Linkage 
“backwards”  into  reviews,  refs,  articles 

•  Version  control 

-  keep  track  of  change  rationale  &  dates 

-  keep  old  version  available 

-  control  national  vs  local  modifications 

*  what  types  of  changes  would  lead  a  national  sponsor 
disavow  a  guideline’s  local  modification? 


A  few  more 

•  Recommendations  need  to  be  auditable 

•  Allow  different  views  for  different  users 

•  specialist,  nurse,  inpt  vs  outpt 

•  consider  wide  variety  of  stakeholders,  users 

•  Must  be  able  to  visualize  GL 

•  Need  a  middle  level  for  transportable 
implementation 

-  like  level  B  computable  GLIF 


Still  More 

•  Represent  what  is  “standard”  versus 
“guideline”  versus  “option” 

•  Represent  places  where  local  users  can  or  should  make 
one  or  another  design  choice,  e.g.  v^ich  diagnostic  test 
their  institution  will  use 

•  Handle  concurrent  multiple  GLs 

•  GL  representation  can  invoke  2  or  more  GLs 

•  Reconcile  competing  or  conflicting  recommendations 

-  maybe  not  reconcile  but  at  least  identify  conflicts 


Issues  external  to  GL 
representation 

•  Reconciliation  of  multiple  GLs 

•  Set-up  and  acquisition  of  critiques 

•  Other  issues  that  can  be  left  external  to  GL 

-  the  method  for  obtaining  patient  preferences 

-  the  outcomes  or  care  process  measures  for  QI 


Boundaries  for  GLs 


What  we  really  need! 


Standardized  vocabulary  of  weasel  words 
Meta-dictionary  of  qualifiers  and 
disclaimers 

A  heuristic  approximation  of  what  the  local 
guru  really  does  when  stumped  by  a  patient 
Maybe?  Peut  etre? 

Go  look  it  up! 


Work  Plan  for  Discussion 


Ontputs  or  Product 


•  Use  this  conference  list  as  seed 

•  Generate  more  candidate  requirements 

•  Reviewing  existing  GL  implementation 
efforts 

-  IHC,  Duke,  GHC,  Kaiser,  more  active  health  plans,  others 

-  GLs  at  different  stages  of  the  life  cycle  diagram 

•  Refine  by  setting  boundaries  &  priorities 

•  Write  draft  of  “requirements” 


white  paper 

-  catalog  of  core  requirements  at  each  level 

-  practicality?  General  user  need? 

-  What  must  be  done  first,  for  version  1 ,0? 
Categorize  Requirements 

-  Authoring  &  Development,  e.g.  thru  GLIF-lvl  A 

-  Implementation  &  Usability,  e.g.  GLIF  levels  B,C 

-  Quality  &  Safety,  applicable  to  some  or  all  stages 


People 


Show  me  the  money! 

-  Cuba  Gooding 

Give  me  the  time! 

-  Peiry  Miller 


Models  &  Representation 

John  Gennari 
John  Fox 
Paul  DeClercq 
Domenico  Pisanelli 
Gunther  Schadow 


What  happened 

•  PROforma  description  (Fox  - 1 0  min) 
-With  cross-fire  and  lively  discussion  (20  min) 

•  Primitives  vs.  PSMs  (de  Clercq  - 10  min) 

-  Wth  cross-fire  and  lively  discussion  (20  min) 

•  Ontologies  (Pisanelli  - 10  min) 

-Wth  cross-fire  and  lively  discussion  (20  min) 

•  HL7’s  RIM  (Schadow  - 10  min) 

-  Wth  cross-fire  and  lively  discussion  (20  min) 


What  didn’t  happen 

•  No  discussion  of  the  GLIF  model 

-  GLIF  2.0  has  well-known  limitations 
-GLIF  3.0  is  still  unknown 

-  Did  discuss  three  levels:  Visualization, 
Representation,  Implementation 

•  No  discussion  of  “best”  models  or  “best" 
representations 

-  Too  early  (or  no  interest?) 

-  Did  critically  compare  different  approaches 


Questions  for  modelers 

•  How  abstract? 

•  What  are  the  appropriate  primitives? 

-  Criteria  &  Steps:  action,  branch,  condition 
-Action,  Decision,  Inquiry,  Plan 
-Actions 

-Observation  types:  definition,  event,  order, 
goal 

•  How  to  compare  representations? 

•  How  do  the  three  levels  interact? 


Next  tasks 

•  Input  from  other  breakouts: 

-  Requirements  group  (level  3) 

-Tools  group  (level  1) 

•  Prioritize:  Which  problems  are  most 
solveable? 

•  Relationship  to  GLIF3: 

-  Reasoning  behind  GLIF3  representational 
choices?? 

-An  opportunity  for  feedback! 


A  challenge  for  GLIF  3 

•  What  are  the  new  set  of  primitives? 

-  Iteration,  case,  events,  patient  state, 

exceptions 

-Are  these  the  right  ones? 

•  What  is  the  “layered”  approach,  and 
how  is  HL7  RIM  incorporated? 

•  What  is  the  representation  language  for 
expressions  (criteria)? 


GLIF  review 

Models  &  Representation 

Second  session 

•  Aziz  bravely  stood  forth  describing 

GLIF3  details 

-  Steps 
-Actions 

-  Decision 

-  Events/exception 

GLIF3  issues 

Evaluating  (validation)  models 

•  Duration  of  activities? 

•  Big,  difficult  challenge 

•  Goals  are  missing 

-  Keep  us  honest,  grounded 

•  Growth  of  a  standard:  subclassing  and 

-  What  set  of  examples?  What  work  setting? 

policies  for  adoption 

•  Toy  problems 

•  Need  for  workflow  management  issues 

•  Big  problems  (for  scalability) 

-  High  cost  of  evaluation 

-  Evaluating  models  vs.  evaluating  ultimate 
goal  (sharing  guidelines,  looking  at  all 
three  levels) 

Top  down,  bottom  up? 

Next  steps 

•  Motivators  for  getting  work  done: 

•  Developmental  strategy: 

-  Next  meeting? 

-  Start  from  cases,  and  test  as  you  go 

-  Publication  opportunities  (special  issue  of 

-  Start  from  theory,  first  principles,  and  test 

JAMIA?) 

as  you  go. 

•  Develop  a  guideline  modeling  community: 

•  Can  we  do  formal  theoretic  analyses  of 

-Web  site,  including  repository 

models  for  guidelines? 

•  Example  guidelines 

•  White  papers 

•  Documentation 

-  Mailing  list 

-  Competitions  (comparisons) 

in  Clinical  Trials 
ReporHrom  Breakout  Session  1 

Carol  Brovemt^^ast  Track  Systems) 
Alexa 

Guideline 

3-4, 

Boston, 


Breakout  (>ganization 

•  1 1  participants,  including  2  moderal^^  1  recorder 

•  Stanford,  UCLA,  NLM,  Fast  Track  S^^^^DSG, 
First  Consulting,  Partners  Healthcare, 

College  of  London,  University  ofPavia 

•  Strategy: 

-  Clinical  trials  is  a  domain  area  that  cuts  across  other^^f 
breakout  sessions 

-  Friday:  Identify  issues  specific  to  clinical  trials 

-  Saturday  breakout:  raise  issues  in  other  breakouts  ■ 


DistinguishifrgXharacteristics: 
Clinical  Trials  (CT)^tei:^s  Guidelines 

•  Commonality: 

-  CT  versus  guidelines?  Subset?  $uperset?ll^^tion? 
Specialization? 

-  Many  common  elements  that  vary  in  degree 
importance  (decision-making,  temporal  issues) 

•  Distinguishing  characteristics: 

-  Strong  scientific  design 

-  Complex  visit,  intra-visit-tasks,  inter-visit  tasks  schec^H 

-  Randomization  (patients,  providers,  hospitals) 

-  Data  gathering  critical;  requirements  are  regulated  ■ 

-  Workflow  compliance  is  regulated  ■ 


Distinguishlrtg^^aracteristics 

(continued) 

•  Strict  eligibility  criteria 

•  More  prescriptive  nature 

•  Pre-anticipated  workflow  deviations  ana]^fc|^ 
deflned  exception  handling  mechanisms 

•  Cycles,  subguidelines,  iteration,  looping,  retnH| 
delays  (especially  in  oncology) 

•  More  granular  detail  V 

•  Revisions/araendments/versioning  are  a  given  ■ 

•  Double-blind  studies  impose  requirements  on  1 

“exposure”  of  protocol  detail  I 


Distinguishirtg-Qharacterisitics 

(continued) 

•  Different  views:  sponsor,  site,  patieh^^ 

•  Coordination  intra-site  and  inter-organ^fcj^i 

•  Sponsor  oversight/monitoring  of  perfomu^^B^ 
sites  (especially  industry) 

•  Patient/subject  view  needed  also 

-  Clinical  trial  registries  that  are  aimed  at  patients 

-  Instructions  to  subjects  \^dlo  are  on-study  V 

•  Interim  analyses  required  during  lifetime  of  studyl 


Clinical  TrfelsL  Lifecycle 


•  Identified  Clinical  Trial  Lifecycli%f^es  with 
different  tasks  and  requirements 

-  Trial  inception  (an  idea) 

-  Trial  design  ^ 

-  Trial  conduct/execution 

-  Trial  data  collection  (sometimes  separate) 

-  Trial  data  analysis 

-  Trial  findings  feed  into  new  trial  ideas/designs 

-  Trial  data  meta-analysis 

•  Presentations  on  trial  design  and  NCI/CII 

-  Jeremy  Wyatt  (UCL)  and  John  Silva  (NCI) 


Task-DrivfeFkRequirements 


~'4^ues  in  Clinical  Trials 

Repotl'ftom  Breakout  Session  2 

Carol  Brovera^l^ast  Track  Systems) 
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Boston, 


•  A  representation  is  not  self-prochuning 

•  Applications/Tasks  determine  requiH^nts 

•  Approach:  enumerate  tasks  per  life-cy^^^ 

•  Some  tasks  during  trial  design  3 

-  Formulation  of  trial,  documentation  of  sources 

-  Statistical  validation  (power  calculation) 

-  Eligibility  criteria  specification 

-  Accrual  simulation 

-  Ethical  review/IRB  (track  communication) 


T  ask-DriverTRqquirements 

•  Some  tasks  during  trial  authoring^^ 

-  Authoring  is  a  “supplier”  to  diflerent  appnMj|t^use 

-  Analysis  of  user  base  and  user  goals  require^^^^- 

-  Logical  model  to  support  authoring  needed 

-  Needs  to  be  intuitive  to  clinician/knowledge  engt^f 
but  must  be  able  to  capture  what  is  needed  in  level* 

-  Mapping  between  different  models  needed  (level  l/a 

-  Trial  documentation  exchange/reuse  of  text 

-  Trial  monitoring/compliance 

-  Trial  data  collection 


T  ask-DrivenRequirements 

'  Some  tasks  during  trial  accrua^W 

-  Matching  a  patient  to  a  set  of  trials 

-  Matching  a  trial  to  set  of  eligible  patiei^HB 

-  Formal  eligibility  determination 

•  After  Informed  Consent  obtained  ^ 

-  Connectivity  to  an  electronic  patient  record  m 
ancillary  systems  (e.g.;  lab,  radiology)  is  an 
implied  prerequisite  for  precision 


T  ask-DrivaFuRequirements 


Task-DnvfeFLRequirements 


Some  tasks  during  trial  conducubnecution 

'  Monitoring  visit  workflow  of  individuS^^nts  on  trial 

-  Monitor  visit/tasks  of  individual  patients 

-  Support/Monitor  data  collection  of  individua^^^^^^ 

-  Inter-visit  tasks  and  reminders 

-  Generate  CRTs  (Case  Report  Forms) 

-  Aggregate  trial  management  within  a  site  (CC/site)^^H 

-  Aggregate  trial  management  across  sites  (CRA/^n9| 

-  Adverse  event  monitoring/reportiog 

-  Resource  management  V 

-  Communication/coordination  (intrasite,  site  to  sponsorjB 


•  More  tasks  during  trial  conduct7&%ecution 

-  Monitoring  visit  workflow  of  individum^^nts  on  trial 

-  Support/Monitor  data  collection  of  indivim^^l^(CRF) 

-  Inter-visit  tasks  and  reminders 

-  Aggregate  trial  management  within/across  site(l^^^^^ 

-  Adverse  event  management/monitoring/reporting^^^H 

-  Resource  management 

-  Communication/coordination  (intrasite,  site  to  spons^^l 

•  Tasks  during  trial  data  analysis  (deferred) 

•  Tasks  during  trial  meta-analysis  (deferred)  ■ 
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Additional  Considerations 


Going  Forward... 


•  Eligibility  criteria  only  hold  until  eltollment: 
separate  from  any  “applicability  critenfc|^t  must 
hold  at  other  points  in  lifecycle  (M.  Pel^^^^^ 

•  Requirements  must  consider  other  differem^^^^ 
designs  besides  randomized  trials  (R.  Lacso^^^* 

-  Prospective  cohort 

-  Multi-arm,  cross-over 

•  Consider  requirements  of  trials  per  different 
disease/treatment  areas 

-  E.g.;  cycles  are  idiosyncratic  to  oncology 


NexTS^ps 

•  Further  refinement  to  requirem^lte 
functional  areas  would  come  from^ 
detailing  per  life-cycle  stage 

-  Functional  requirements 

-  Representation  requirements 

-  Infrastructure  requirements 


More  DetailediR^uirements 

•  Further  work  on  functional  reqJJlf^ents 

-  E.g.;  what  is  the  complete  set  of  tasra^^^ow 
to  prioritize  them? 

•  Some  special  representation  needs? 

-  E.g.;  data  collection  visits  and  CRFs 

-  Time  windows  for  protocol  visits/tasks 

-  Uncertainty 


More  Detailedi^^uirements 

•  What  special  infrastructure  reqfHKments? 

-  Authoring  tool  requirements 

-  Dissemination  of  amendments 

-  Versioning  control/maintenance 

-  Others? 


P1-2:  Problems^tl^ed  in  Priority) 

•  Identify  users/stakeholders 

-  Expectations  and  requirements;  solicit  inp^^^ 

•  Evaluate  existing  standards  and  requiret^^Bfiu: 
protocol  content  and  reporting 

•  Evaluate  and  extend  exisiting  representation^^^ 

•  Create  research/consortium  testbed 

-  E.g.  Protocols  authored  in  “standard”  representation, 
sample  test  patient  data.... 

•  Build  demonstration  systems  that  use  testbed 

•  Promote  shareability  and  collaboration 
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T  echnology/syntax/  structure 


Iterative  refinement 


Interoperable 

Integratable 

Portable 

Maintain/evolve/survivable 
Computational  scalability 


•  version  control 

•  reason  for  revision,  justification 

•  allow  multiple,  independent  development 

•  conflict  identification 

•  reconciliation 

•  support  metrics  for  detemining  effect  of 
change 


Plan  for  today 

Further  refinement  of  high  level 
requirements 


V 


4  P’s 

•  Problems 

•  Priorities 

•  Planning 

•  People 


Session  2:  Change  of  focus 

m 


Authoring  Tools 


Block  diagram 


•  May  need  to  support  multiple  representation 
schemes 

•  Support  multiple,  concurrent  authorship 

•  Connection  to  repository/libraries 


Big  Picture  of  Tools  & 
Infrastructure 


Authoring  Tool 

Composing  tool  w  links  to 
repository  of  building  blocks 
*  Update  Tools 

Conflict  resolution 

Terminology  Management 
Feasability  Tool 

Verification  Tool 

Multiple  levels  of  granularity 
Multiple  ontologies 

Next  Steps 

•  Extend  the  Set  of  Tools,  Their  Definition,  Key 
Components  and  Services 

•  Probiem  -  How  to  DOiT? 

•  What  are  the  High  Leverage  Points  for  investment? 

•  What  are  the  ESSENTiAL  Experiments 

•  EVALUATE,  EVALUATE  and  FEEDBACK 


Building  Infrastucture 

How  to  address  the  diverse  requirements  and  Issues? 


•  Bootstrap 

-  Not  top-down 

-  Not  a  traditional  St  engagement 

-  MuRipie  atak^older  participation 
from  tha  outset 


a  Define  Mrvice  interfaces 

-  Examples:  CDEs,  CRF  tempMes 

-  Critical  commonalities 

-  Highly  leveraged:  basis  for  scaling 

-  Minimal  and  separable 

-  Concrete 

-  Avoid  Implementation  bias 


Diversity  of 
service  clients 


Service  interface 


Diversity  of 
service  providers 


e  Foster  diversity 

-  DIvarsHy  of  service 
providers 

-  Diversity  of  clients 

-  Areas  for  innovation 
and  competition 


Process  and  People 


•  Internet  "RFC”  model 

-  Commonality  +  Prototypes 

-  Minimality  and  focus 

•  Experiments  at  scale 

-  Dogfbod 

•  Ongoing  evolution  and  scaling  up 


Organization  and  Process 

Breakout  Session  1  Summary 

Breakout  Session  1 

•  10  participants 

March  3, 2000 

•  Issues  addressed 

-  Alternative  organizational  structures 

-  HL7:  advantages  and  disadantages 

Breakout  Leader: 

-  Stakeholders 

John  Dulcey,  M.D. 

-  Next  Steps 

Recorder:  Robert  Greenes,  M.D. 

Alternative  organizational  structures 

•Form  an  independent  non-profit 
consortium 

•Align  with  an  existing  SDO  for  specific 
components  of  guideline  development 

•Become  a  subgroup  of  an  existing  SDO 

•Create  a  proprietary  entity  to  seed  the 
industry  (Eg.  WAP) 


Advantages  of  HL7 

•  Has  high  visibility 

•  Has  industry  participation 

•  Is  the  sponsoring  organization  for  Arden 
Syntax 

•  Has  an  established  schedule  of  working 
group  meetings 

•  Has  significant  international  representation 

•  Has  an  established  development  methodology 


Disadvantages  of  HL7 

•  Extra  meetings  for  some 

•  Price  of  membership  and  meeting  fees 

•  Not  coincident  with  main  mission  of  HL7 

•  Not  fully  international  in  scope 


Stakeholders  in  Guideline  Process 

•  Developers  of  content  of  guidelines 

•  Developers  of  tools  for  guideline  authoring 

•  Guideline  standards  developers 

•  Application  integrators 

•  Guideline  users 
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Next  Steps 

•  Form  a  consortium  of  academia  and  industry 
leaders 

•  Develop  a  business  plan  that  includes  financial 
support  for  several  full  time  staff  to  develop 
guidelines 

•  Look  into  possible  industry  and  governmental 
support  fo  finance  a  guideline  development 
organization 

•  Identify  participants  in  initial  task  force  to 
move  the  process  forward 
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Organization  and  Process 

Strategic  Planning 

Breakout  Session  2 

March  4, 2000 

•Vision 

•Mission 

Breakout  leaders: 

•Values 

John  Dulcey,  M.D. 

Peter  Elkin,  M.D. 

•Goals 

Recorder:  Robert  Greenes,  M.D. 

•Objectives 

Vision 

Mission 

To  create  a  global,  sharable  framework 
for  guideline  development 
and  implementation. 

Enable  the  widespread  distribution  of 
health  knowledge  to  improve  health 
status,  improve  quality  and  efficiency  of 
healthcare,  and  reduce  health  care  costs. 

-  Improve  health  care  and  quality  of  life  of 
world  citizens  through  better  use  of  medicai 
knowiedge 

-  Global  use  of  a  consistent  framework  that 
transitions  between  guideiine  development 
and  implementation. 

Framework 

•  Standards  Development 

•  Mechanism  for  stimulating  the 
creation  and  widespread  distribution 
of  guidelines 

•  Forum  for  exchange  of: 

—  Ideas 

-  Public  domain  core  products 


Business  Plan 

•  Understanding  the  Problems 

-Needs 

-  Solutions 

•  Understanding  the  Opportunities 

—  Time  constraints 

-  Potential  liaisons 


Appendix  D 

Functional  requirements  for  a  shared  guideline  representation 
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Shared  guideline  representation 


Functional  requirements  for  a  representation  for  sharable 

guidelines 


Aziz  A.  Boxwala,  M.B.B.S.,  Ph.D.',  Mor  Peleg,  Ph.D.^  Robert  A.  Greenes,  M.D.,  Ph.D.*, 
Edward  H.  ShortUffe,  M.D.,  Ph.D.^’^  Vimla  L.  Patel,  Ph.D.^ 

'Decision  Systems  Group,  Harvard  Medical  School,  Brigham  &  Women’s  Hospital,  Boston,  MA 
^Stanford  Medical  Informatics,  Stanford  University  School  of  Medicine,  Stanford,  CA 
^Center  for  Medical  Education,  McGill  University,  Montreal,  PQ,  Canada 
'’Department  of  Medical  Informatics,  Columbia  University,  New  York,  NY 


Recent  trends  in  health  care  dehvery  have  led  to  an  increased  emphasis  on  the  development  of 
guidelines  for  prevention,  diagnostic  work-up,  treatment,  and  patient-management  processes. 
Such  guidelines  are  motivated  by  concerns  about  marked  variations  in  clinical  practice  and  are 
designed  to  help  to  provide  a  common  standard  of  care  both  within  a  health  care  organization 
and  among  different  organizations.  For  better  integration  of  guidelines  into  the  clinical 
workflow,  they  are  increasingly  being  disseminated  and  implemented  using  computer-based 
systems.  The  range  of  possible  appUcations  for  computer-based  guidelines  is  very  broad, 
including  use  for  disease  management,  encounter  workflow  facilitation,  reminders/alerts,  design 
and  conduct  of  clinical  trials,  care  plan/critical  path  support,  appropriateness  determination,  risk 
assessment,  demand  management,  education  and  training,  and  reference. 

Computer-based  approaches  to  representation  of  guidelines  are  being  developed  by  various 
groups  [1-7].  Critical  to  sharing  of  the  knowledge  in  these  guidelines  across  institutional, 
national,  and  medical  domain  boundaries  would  be  adoption  of  a  common  format  for 
representing  them.  In  order  to  be  widely  usable  and  acceptable,  such  a  common  representation 
for  guidelines  must  provide  several  functional  capabilities.  The  representation  must  accoimt  for 
requirements  for  (a)  human  communication,  (b)  validation  of  logical  consistency  and 
completeness  (not  correctness),  and  (c)  incorporation  into  institutional  information  system 
environments.  We  hereby  propose  a  set  of  functional  requirements  for  a  sharable  guideline 
representation  language  and  briefly  provide  their  underlying  rationale: 

1.  Support  for  different  types  of  guidelines.  Guidelines  may  be  classified  [8]  along  a  variety  of 
axes  such  as:  (a)  stage  of  the  care  process,  e.g.,  screening,  diagnostic  workup,  and  treatment; 
(b)  the  medical  domain;  (c)  the  intended  application,  such  as  in  disease  management,  critical 
paths/care  plans,  clinical  trial  conduct,  and  appropriateness  determination.  Different  types  of 
guidelines  may  entail  representations  of  fundamentally  different  concepts.  For  example,  an 
appropriateness  criterion  that  evaluates  relative  suitability  of  two  diagnostic  tests  would 
represent  a  decision  making  process.  A  clinical  trial  protocol,  on  the  other  hand,  represents  a 
prescriptive  patient  management  plan  that  contains  concepts  such  as  treatment  visit  and 
adverse  event.  The  guideline  representation  format  must  be  flexible  to  accommodate  these 
needs. 

2.  Different  modes  of  use.  Encoded  guidelines  can  potentially  be  used  in  different  modes.  Users 
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may  read  or  browse  g;uidelines  as  educational  and  reference  resources.  Guidelines  could  be 
used  interactively  for  patient-specific  decision  support  and  workflow  support.  Quality 
assurance  applications  would  use  guidelines  as  benchmarks  of  quahty  care,  perhaps  in  a 
batch-processing  mode.  The  knowledge  in  guidelines  must  be  represented  in  a  format  that  is 
independent  of  the  expected  mode  of  use.  The  representation  must  enable  the  variety  of  uses 
by  structuring  the  knowledge  in  a  way  that  will  support  its  retrieval  for  all  those  likely  uses. 

3.  Adaptation  of  guidelines  for  local  use.  Due  to  variations  in  health  care  settings,  guidelines 
developed  by  national  organizations,  medical  specialty  organizations,  or  under  other  broad 
aegis  often  need  to  be  modified  before  practitioners  find  them  suitable  for  local  use.  Reasons 
for  adaptation  of  guidelines  include  variations  among  settings  due  to  the  type  of  institution 
(e.g.,  hospital  vs.  office),  location  (e.g.,  urban  vs.  rural),  differential  availability  of  equipment 
and  medications,  dissimilarity  of  patient  population  (e.g.,  as  reflected  in  prevalence  of  the 
disease),  and  local  policies  and  workflow  patterns.  A  common  representation  format  must 
provide  the  ability  to  adapt  knowledge  contained  in  guidelines,  and  track  and  document 
modifications  to  the  guideline. 

4.  Integration  with  institutional  systems.  For  integrating  guidelines  into  the  clinical  workflow, 
references  to  patient  data  and  clinical  actions  in  guidelines  will  need  to  be  mapped  to  their 
instantiations  or  implementations  in  heterogeneous  clinical  mformation  systems 
environments.  This  requirement  implies  the  use  of  standard  vocabularies  and  standard 
reference  data  models  by  the  guideline  representation  format,  which  mapping  tools  can 
utilize  to  achieve  the  integration. 

5.  Revision  tracking.  Guidelines  are  often  revised  in  response  to  changing  medical  knowledge. 
The  representation  format  must  keep  track  of  and  document  revisions  to  the  guideline. 
Among  other  reasons,  revision  tracking  is  important  for  incorporating  new  versions  of 
externally  developed  guidelines  into  institutional  use. 

6.  Managing  complexity.  Guidelines  and  their  logic  may  get  fairly  complex.  The  representation 
format  must  deal  with  this  complexity  by  abstracting  details  into  high-level  concepts.  The 
management  of  complexity  in  representation  is  important  during  authoring  and  viewing  of 
guideline. 

The  InterMed  project,  a  collaboration  among  medical  informatics  groups  at  Stanford,  Harvard, 
McGill,  and  Columbia  universities,  supported  by  the  National  Library  of  Medicine,  developed  a 
representation  for  sharable  clinical  guidelines.  The  initial  version  of  the  representation,  known  as 
the  Guideline  Interchange  Format  (GLIF),  was  published  in  1998.  A  follow-on  project,  fimded 
by  both  the  NLM  and  the  US  Army,  brings  together  investigators  at  Harvard,  Stanford, 
Columbia,  and  McGill,  with  guideline  developers  from  the  American  College  of  Physicians- 
American  Society  of  Internal  Medicine,  to  further  define  guideline  representation  requirements 
so  as  to  facilitate  authoring,  sharing,  and  integration  into  applications.  The  requirements  stated 
above  are  a  result  of  our  experience  with  the  development  and  use  of  GLIF,  and  form  the  basis 
for  refinements  being  made  to  the  GLIF  specification. 
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reexamined  the  need  for  the  limitation  assigned  to  technical 
reports  written  for  this  Command.  Request  the  limited 
distribution  statement  for  the  enclosed  accession  numbers  be 
changed  to  "Approved  for  public  release;  distribution  unlimited. 
These  reports  should  be  released  to  the  National  Technical 
Information  Service. 

2.  Point  of  contact  for  this  request  is  Ms.  Kristin  Morrow  at 
DSN  343-7327  or  by  e-mail  at  Kristin.Morrow@det.amedd.army.mil. 

FOR  THE  COMMANDER; 


End  PHYLIsS  M.  RINEHART 

Deputy  Chief  of  Staff  for 
Information  Management 
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Vs,'. 


